From 4c64f76adb19152a70961c5bf7a049c104046cd3 Mon Sep 17 00:00:00 2001 From: tweinmann Date: Wed, 28 Oct 2026 19:58:39 +0000 Subject: [PATCH 1/3] Propose phase 1: stacks and routing (v0.1) Proposal, delta specs, design and tasks for setup, up, status, ps, logs, open, down, skill and hostname and path routing. Co-Authored-By: Claude Opus 5.5 --- .../phase-1-stacks-and-routing/.openspec.yaml | 1 - .../phase-1-stacks-and-routing/design.md | 204 ++++++++++++++++++ .../phase-1-stacks-and-routing/proposal.md | 77 +++++-- .../specs/agent-skill/spec.md | 26 +++ .../specs/cli-contract/spec.md | 72 +++++++ .../specs/dev-environment/spec.md | 21 ++ .../specs/proxy/spec.md | 67 ++++++ .../specs/routing/spec.md | 89 ++++++++ .../specs/stack-identity/spec.md | 101 +++++++++ .../specs/stack-lifecycle/spec.md | 125 +++++++++++ .../specs/stack-list/spec.md | 51 +++++ .../specs/stack-status/spec.md | 85 ++++++++ .../phase-1-stacks-and-routing/tasks.md | 71 ++++++ 13 files changed, 966 insertions(+), 24 deletions(-) create mode 100644 openspec/changes/phase-1-stacks-and-routing/design.md create mode 100644 openspec/changes/phase-1-stacks-and-routing/specs/agent-skill/spec.md create mode 100644 openspec/changes/phase-1-stacks-and-routing/specs/cli-contract/spec.md create mode 100644 openspec/changes/phase-1-stacks-and-routing/specs/dev-environment/spec.md create mode 100644 openspec/changes/phase-1-stacks-and-routing/specs/proxy/spec.md create mode 100644 openspec/changes/phase-1-stacks-and-routing/specs/routing/spec.md create mode 100644 openspec/changes/phase-1-stacks-and-routing/specs/stack-identity/spec.md create mode 100644 openspec/changes/phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md create mode 100644 openspec/changes/phase-1-stacks-and-routing/specs/stack-list/spec.md create mode 100644 openspec/changes/phase-1-stacks-and-routing/specs/stack-status/spec.md create mode 100644 openspec/changes/phase-1-stacks-and-routing/tasks.md diff --git a/openspec/changes/phase-1-stacks-and-routing/.openspec.yaml b/openspec/changes/phase-1-stacks-and-routing/.openspec.yaml index 0d31f8b..1bb6456 100644 --- a/openspec/changes/phase-1-stacks-and-routing/.openspec.yaml +++ b/openspec/changes/phase-1-stacks-and-routing/.openspec.yaml @@ -1,3 +1,2 @@ schema: spec-driven created: 2026-10-28 -skip_specs: true diff --git a/openspec/changes/phase-1-stacks-and-routing/design.md b/openspec/changes/phase-1-stacks-and-routing/design.md new file mode 100644 index 0000000..fda8d42 --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/design.md @@ -0,0 +1,204 @@ +# Design + +## Context + +Phase 0 left a Cobra root with `version`, the document types in `internal/cli/json.go`, an +`Error` type with a stable code, golden tests, and a shell smoke test showing that the building +blocks work: `ports: !reset []`, Traefik with the Docker provider on the network `siding`, Host +and PathPrefix rules (`hack/smoke/testdata/override.yml` is the hand-written model of what +`siding up` must generate). Motivation and scope: see proposal.md; behaviour: see the delta +specs. Constraints from CLAUDE.md: Docker is the database, the compose file is never edited, no +published ports, runtime truth, the same output for agent and human, no package-level test +seams. + +## Goals / Non-Goals + +**Goals:** +- Every package that touches the machine (git, `docker compose`, Docker API, HTTP probe, clock, + browser) sits behind a small interface in `cli.Options`, so all commands, verdicts and + overrides are tested without Docker, in parallel, against golden files. +- One end-to-end test of the real thing against the inner daemon (`just smoke-stacks`). + +**Non-Goals:** +- Acting on a stack other than the current worktree's (`down `, `gc`): v0.2. +- Memory budget, exit 3, web overview: v0.2. Snapshots, seeds: v0.3. Idle stop: v0.4. +- Reaching a published database port from the host: ports are reset; `docker compose exec` + or a routed admin UI is the way in v0.1. +- Path stripping, TLS, WebSocket specifics: Traefik passes WebSockets through as it is. + +## Decisions + +### D1. Package layout +`internal/git` (worktree, branch, common dir, dirty, merged), `internal/compose` (run +`docker compose`, parse `config --format json`), `internal/docker` (Engine API: containers, +networks, volumes, logs, stats), `internal/stack` (identity, slugs, labels, override +generation, routing table: pure functions), `internal/verdict` (from service states and probe +results to a verdict: pure), `internal/proxy` (the embedded proxy compose file, setup), and +`internal/cli` for commands and documents. Pure packages get table-driven tests; the others are +thin and covered by the end-to-end test. + +### D2. Reading the user's compose file through `docker compose config` +siding runs `docker compose -f config --format json` and reads services, ports, +`expose`, labels, networks, `network_mode`, healthchecks, volumes and `env_file` from the +normalised result. Compose does interpolation, profiles, `include` and `extends`; siding parses +none of it itself. Alternative: the compose-go library in process. Lost: it would pin a second +Compose implementation into the binary that can disagree with the user's `docker compose` +(principle 8, standard building blocks). + +### D3. Docker Engine API through `github.com/moby/moby/client` +State (containers by label, health, exit codes, published ports, logs, one-shot stats) is read +through the Engine API. `github.com/docker/docker` is frozen at v28; the client now lives in +its own module. Lifecycle (`up`, `down`, the proxy) goes through `docker compose`, as the +conventions say. Alternative: parsing `docker inspect` output. Lost: slower, and more text +formats to keep stable. + +### D4. Proxy as Compose project `siding` (maintainer's decision: port 80, `--port`) +`siding setup` writes the embedded compose file to `~/.siding/proxy/compose.yml` (a losable +cache, rewritten on every setup) and runs it as project `siding`, container `siding-proxy`, +image pinned in the binary (`traefik:v3.7.16`, the same as the devcontainer's +`SIDING_TRAEFIK_IMAGE`), published on `127.0.0.1:`, port 80 by default. Traefik flags: +Docker provider, `exposedbydefault=false`, `network=siding`, and a constraint +`LabelRegex(\`dev.siding.repo\`, \`.+\`)`, so foreign Traefik labels are ignored. The port is +read back from the container's port binding (Docker is the database); no file holds it. +In the devcontainer the developer runs `siding setup --port 8080`, the forwarded port. + +Alternatives for the port, asked: 8080 by default (every URL carries `:8080`); 80 with an +automatic fallback to 8080 (convenient, less predictable). Alternative for the mechanism: +a plain container created through the API. Lost: the proxy then is not a Compose project, +and "removable, plain Compose" (principle 9) would not hold for it. + +### D5. Identity (maintainer's decision: repository name from the main worktree) +Repository name = basename of the parent of `git rev-parse --path-format=absolute +--git-common-dir`, overridable with `--repo`. Slugs as in the spec `stack-identity`; 40 +characters keep `...localhost` labels well under DNS's 63 and project +names short. Alternative, asked: the name of the `origin` remote (stable when the folder is +renamed, but needs a remote and differs from what the user sees in the file system). + +A detached HEAD is an error (`NoBranch`), because a slug from a commit would make a new stack +for every commit. Collisions (`feature/a`, `feature-a`) are detected through the labels of the +existing project, not prevented by hashing every slug, so normal names stay readable. + +### D6. Machine id +`sha256("siding-machine-id:" + os id)`, first 16 hex digits; the os id is `/etc/machine-id` +on Linux and `IOPlatformUUID` from `ioreg` on macOS. Without one (many containers), a random id +in `~/.siding/machine-id`. Hashing keeps the raw id, which other software treats as +confidential, out of labels. Losing the file changes the id; in v0.1 that only changes a label +nobody filters on yet, and v0.5 can make the fallback durable if remote hosts need it. +Alternative: always a generated file. Lost: reinstalling or clearing `~/.siding` would change +the id even where the OS has a stable one. + +### D7. The override (maintainer's decision: `dev.siding.port` for ambiguous ports) +Generated from the normalised config, written as YAML with `!reset` tags: +- every service: `ports: !reset []`, labels `dev.siding.*`; +- every HTTP service not hidden: `networks:` = its normalised networks plus `siding` (a list + in an override replaces the default network otherwise), `traefik.enable=true`, + `traefik.docker.network=siding`, one router `-` for + `Host(\`...localhost\`)`, and for a route path a second router + `--route` for `Host(\`..localhost\`)` plus + `PathPrefix(\`\`)` (none for `/`); Traefik's default priority (rule length) makes the + longest prefix win; +- a headers middleware on every siding router that adds `X-Siding-Service: /` + to responses (D9); +- top-level: labels on the project's own networks (`default` included) and named volumes, not + on external ones; the network `siding` as external. +Golden files under `internal/stack/testdata/override/` hold input config and expected override. + +HTTP detection, asked: exactly one container port from `ports` targets and `expose`, or +`dev.siding.port`. Alternatives: the first port wins (a database with a published port would +be routed by accident); only services with a route label (drops the service host of the +roadmap). The new label changes the convention list in CLAUDE.md and `openspec/config.yaml`. + +A path prefix is passed on unchanged (no StripPrefix): the common dev setups (Vite, nginx, +Next.js rewrites) mount an API under `/api` and expect `/api` to arrive. A stripping option +can be added later as a label value without breaking this one. + +### D8. `.siding/.gitignore` +siding writes `.siding/.gitignore` with `*`, which ignores the directory including itself, so +`.siding/` never makes the worktree dirty and the user's `.gitignore` is never touched. +Alternative: ask users to add `.siding/` to `.gitignore`. Lost: one more setup step, and a +forgotten one would make `down --volumes` refuse with `DirtyWorktree`. + +### D9. Readiness probe through the proxy +siding sends `GET ` with `Host: ` to `127.0.0.1:` (no DNS +needed) for every service with a route path, and for `/` routes to `/`. A response counts as +the service's own when it carries `X-Siding-Service` for that service and its status is not +502, 503 or 504. Traefik's own 404 for a missing router has no such header; its 502/503/504 +for an unreachable backend do not count. The probe follows no redirects; a 3xx counts. +Alternatives: any status below 500 (cannot tell Traefik's 404 from the app's); asking +Traefik's API whether the router exists (needs a second published port). The end-to-end test +checks that Traefik really adds the header to backend responses and not to its own errors; +if it adds it to its 502 too, the status check still rules those out. + +### D10. Verdict +A pure function from (services in the config, containers with state, health, exit code, +restart count, probe results, elapsed time, timeout) to the verdict. Order of evaluation: +`Missing` (no container after `up` has finished), `Exited` (non-zero), `Unhealthy`, +`Restarting` (restart count grew by 3 or more while waiting, or state `restarting` at the end), +then `Starting` until healthy, running or exited 0 and answering. `--wait` polls every second. +A service that exits 0 counts as done (migrations, seeds). The first failing service in +compose order is reported; others appear in `services` with their state. The hint is a fixed +text per reason, completed with the exit code or the last healthcheck output. Log tail: 20 +lines from the Docker API, redacted (D12). + +`status` without `--wait` and with `Starting` exits 2: "not ready now" is the same answer as a +wait of zero. Alternative: exit 0 with the verdict in the document. Lost: a script checking +only the exit code would take Starting for Ready. + +### D11. `down` (maintainer's decision: containers go, volumes stay) +`docker compose -p down --remove-orphans` (by name, works without the compose file); +`--volumes` adds `-v` after the guard: `git status --porcelain` empty, and the branch tip an +ancestor of the default branch (`refs/remotes/origin/HEAD`, else local `main`, else +`master`, else unmerged). `--dry-run` lists containers, networks and, with `--volumes`, +volumes by label through the API. Alternatives, asked: `down` always removes volumes and is +always guarded (an agent could not stop its stack in a dirty worktree without `--force`); +`down` only stops (removing would wait for `gc`). This makes the guardrail "never lose work" +precise: it guards a stack's data and worktrees, not containers. CLAUDE.md and +`openspec/config.yaml` change with it. A squash-merged branch counts as unmerged; `--force` +is the way until `gc` (v0.2) learns more. + +### D12. Redaction +Values from the worktree's `.env` and from every `env_file` in the normalised config, 4 +characters or longer, longest first, are replaced by `***` in every log line siding puts into a +document. The `.env` is parsed with Compose's rules for `KEY=VALUE`, quotes and comments; no +values are logged or cached. Shorter values are left, since `***` for `1` or `on` would wreck +every line. Text output of `siding logs` stays raw: it is the user's own terminal, as with +`docker compose logs`. + +### D13. `siding skill` (decided here, open for review) +The skill text is embedded in the binary (`internal/cli/skill/SKILL.md`, golden-tested) and +printed; siding writes no file into the user's repository. Alternatives: `siding skill +install` writing `.claude/skills/siding/SKILL.md` (one more file siding writes into a user's +repository); a Claude Code plugin marketplace entry (an extra channel to keep in step with the +binary). Printing keeps the text and the binary in step and lets the user choose where it goes. + +### D14. Test seams +`cli.Options` gains `Git`, `Compose`, `Docker`, `Probe`, `Browser`, `Clock`, `Home`, `Env` +and `Proxy` (container name, network, project, image, default port). `main` fills them with +real implementations; unit tests use fakes. The end-to-end tests under `internal/e2e` with +`//go:build docker` run commands in process with `Proxy` set to `siding-test-proxy` on the +network `siding-test` at port 18080, in temporary git repositories whose `--repo` starts with +`siding-test-`, and remove everything by exact name in `t.Cleanup`. So the binary has no hidden +test switches, and a developer's `siding-proxy` in the devcontainer is never touched. + +## Risks / Trade-offs + +- [Some tools do not resolve `*.localhost`] → browsers and curl do; siding's own probe uses + `127.0.0.1` with a Host header; README notes `curl` and `/etc/hosts` for others. +- [Port 80 is taken or not bindable] → `PortInUse` with the hint `--port 8080`; addresses then + carry the port. +- [Users lose host access to published database ports] → documented in README with + `docker compose exec`; a routed TCP option is out of scope for v0.1. +- [Traefik adds `X-Siding-Service` to its own errors] → the status check excludes 502/503/504; + the end-to-end test pins the behaviour. +- [`docker compose config` differs between Compose versions] → siding requires 2.24 and checks + the version once per command that runs Compose; golden tests cover the parsed fields. +- [Squash merges count as unmerged] → `--force`, and the hint says so. +- [Redaction misses secrets that are not in `.env` files, e.g. inline `environment:` values] + → the guarantee is stated as `.env` and `env_file` values only, which is what the safety + rule asks; inline values are visible in the compose file anyway. + +## Migration Plan + +Nothing to migrate: v0.1 is the first release with stacks. Uninstalling leaves plain Compose +projects (`-` and `siding`) that keep running and are removed with +`docker compose -p down`. diff --git a/openspec/changes/phase-1-stacks-and-routing/proposal.md b/openspec/changes/phase-1-stacks-and-routing/proposal.md index 751efc9..d9b825e 100644 --- a/openspec/changes/phase-1-stacks-and-routing/proposal.md +++ b/openspec/changes/phase-1-stacks-and-routing/proposal.md @@ -1,36 +1,67 @@ # Proposal -> Draft. This change holds the roadmap text only and is marked `skip_specs: true`. -> `/opsx:propose` turns it into a full change: it writes the delta specs, `design.md` and -> `tasks.md`, revises this proposal, and removes `skip_specs` from `.openspec.yaml`. - ## Why -siding exists to answer "does what my agent built actually run?". v0.1 is the smallest siding -that answers it: one stack per worktree, reachable by name, with a verdict an agent can wait for. +siding exists to answer "does what my agent built actually run?". Today it only prints its +version. v0.1 is the smallest siding that answers the question: every worktree runs its own +stack from the unchanged compose file, every stack is reachable by name through one proxy, and +an agent can wait for a verdict, Ready or Failed with the reason and the logs, as versioned +JSON. ## What Changes -Roadmap v0.1: - -- `siding setup`: the shared proxy `siding-proxy` (Traefik, Docker provider) on the network - `siding`, and the stable machine id. -- `siding up`: writes `.siding/override.yml` (ports reset, network `siding`, Traefik and - `dev.siding.*` labels) and runs the user's compose file as project `-`. -- `siding ps`: every stack on the machine, read from labels and the Docker API. -- `siding status --wait --json`: the verdict Ready, or Failed with service, reason, hint and - the last log lines; exit 0 Ready, 1 Failed, 2 TimedOut. -- `siding logs`, `siding open`, `siding down` (refuses unmerged or dirty work unless forced; - `--dry-run`). -- Hostname and path routing: `..localhost`, `dev.siding.route` (`/`, - `/`, `false`), `...localhost` for unrouted HTTP services. -- A Claude Code skill that uses the `--json` documents. +- `siding setup [--port ]`: starts the shared proxy `siding-proxy` (Traefik, Docker + provider) on the network `siding`, listening on `127.0.0.1:80` by default. Idempotent. +- `siding up [--wait] [-- ]`: in the current worktree, writes + `.siding/override.yml` (ports reset, network `siding`, Traefik and `dev.siding.*` labels) and + runs the user's compose file as project `-`. With `--wait` it ends with the + verdict of `status --wait`. +- `siding status [--wait [--timeout ]]`: the verdict Ready, Starting, Failed or TimedOut; + a failure names the service, the reason, a hint and the last log lines of that service. +- `siding ps`: every stack on the machine, read from labels and the Docker API, with its + state, address and memory use. +- `siding logs [service…]`, `siding open [service]`: the stack's logs and its address. +- `siding down [--volumes] [--force] [--dry-run]`: removes the stack's containers and network. + Volumes go only with `--volumes`, which refuses when the branch is unmerged or the worktree + dirty, unless `--force`. +- Routing: `..localhost` for the stack; `dev.siding.route` (`/`, + `/`, `false`) mounts services on it; every HTTP service also answers at + `...localhost`. A new service label `dev.siding.port` picks the + port of a service with several. +- `siding skill`: prints a Claude Code skill that tells an agent how to start its stack and + wait for the verdict. +- Every command has `--json`, with the new kinds `Setup`, `Stack`, `Status`, `StackList`, + `Logs`, `Address`, `Down` and `Skill`, and new stable error codes. +- The guardrail "never lose work" is made precise: it guards removing a stack's data (its + volumes) or a worktree; removing containers loses nothing and is not guarded. CLAUDE.md and + the project context change with it, as does the convention list (`dev.siding.port`). ## Capabilities -To be decided at `/opsx:propose`. +### New Capabilities +- `cli-contract`: the `--json` documents, the `Error` document, exit codes and `siding version` + (exists since Phase 0, now specified). +- `stack-identity`: how a worktree maps to repo name, branch slug, Compose project, host names, + machine id and the `dev.siding.*` labels. +- `proxy`: `siding setup` and the shared Traefik `siding-proxy` on the network `siding`. +- `routing`: which services are HTTP services, how route labels map to hosts and paths, and + `siding open`. +- `stack-lifecycle`: `siding up`, `siding down`, `siding logs`, the override file and the + guard that keeps unmerged or dirty work. +- `stack-status`: the verdict of `siding status`, its reasons, hints and log tail. +- `stack-list`: `siding ps`. +- `agent-skill`: the Claude Code skill printed by `siding skill`. + +### Modified Capabilities +- `dev-environment`: a second smoke test, `just smoke-stacks`, runs siding's own commands + end to end against the inner daemon, in CI on pull requests and on `main`. ## Impact -Go code under `internal/`, the Docker Go SDK as a dependency, new smoke tests in the -devcontainer's inner daemon. +- Go code under `internal/` (git, compose, Docker, routing, verdict) and new commands in + `internal/cli`; the Docker Engine Go SDK as a new dependency. +- `docker compose` (>= 2.24) and `git` become runtime requirements of siding. +- New golden files per document kind and for generated overrides; an end-to-end test under a + `docker` build tag; `justfile` and CI gain `smoke-stacks`. +- CLAUDE.md, README and `openspec/config.yaml`: status, the precise guardrail, the label + `dev.siding.port`, the proxy port. diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/agent-skill/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/agent-skill/spec.md new file mode 100644 index 0000000..56d4981 --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/specs/agent-skill/spec.md @@ -0,0 +1,26 @@ +# Spec Delta + +## Purpose + +Defines the Claude Code skill siding ships, so that a coding agent starts its worktree's stack +and waits for the verdict instead of guessing whether its work runs. + +## ADDED Requirements + +### Requirement: siding skill prints the skill +`siding skill` SHALL print a Claude Code skill (`SKILL.md` with name, description and body) on +stdout and write no file. With `--json` it SHALL print a `Skill` document holding the same text. + +#### Scenario: Install the skill +- **WHEN** a user runs `mkdir -p .claude/skills/siding && siding skill > .claude/skills/siding/SKILL.md` +- **THEN** the file is a valid skill named `siding`, and siding wrote nothing itself + +### Requirement: What the skill teaches +The skill SHALL tell the agent to run `siding up --wait --json` after a change that affects the +running app, to read the verdict and exit code, to fix the named service from the failure's +reason, hint and log tail, and to report the address when Ready. It MUST NOT tell the agent to +use `--force`, `--volumes` or `siding share`. + +#### Scenario: The commands in the skill +- **WHEN** the printed skill is read +- **THEN** it names `siding up --wait --json`, `siding status --json`, `siding logs --json` and the exit codes 0, 1 and 2, and none of `--force`, `--volumes` or `share` diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/cli-contract/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/cli-contract/spec.md new file mode 100644 index 0000000..5db6ff7 --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/specs/cli-contract/spec.md @@ -0,0 +1,72 @@ +# Spec Delta + +## Purpose + +Defines what agents and scripts can rely on when they call siding: one versioned JSON document +per command with `--json`, stable error codes and fixed exit codes. + +## ADDED Requirements + +### Requirement: One JSON document per command +Every siding command SHALL accept `--json` and then print exactly one JSON document on stdout +and nothing else there. The document SHALL carry `apiVersion: siding.dev/v1alpha1` and a +`kind`. Empty fields MUST be left out, never printed as `null`. + +#### Scenario: A command with --json +- **WHEN** `siding version --json` runs +- **THEN** stdout holds one document with `apiVersion` `siding.dev/v1alpha1` and `kind` `Version`, and nothing else + +#### Scenario: Progress of docker compose +- **WHEN** a command with `--json` runs `docker compose`, which reports progress +- **THEN** that progress goes to stderr, and stdout still holds only the document + +#### Scenario: An empty field +- **WHEN** a document has a field without a value, such as a commit unknown to the build +- **THEN** the field is missing from the document + +### Requirement: Compatible within a version +Within `siding.dev/v1alpha1` a document field and an error code MUST NOT be removed, renamed +or given another meaning; new fields, kinds and codes MAY be added. + +#### Scenario: A field is added +- **WHEN** a later release adds a field to the `Status` document +- **THEN** documents still carry `apiVersion` `siding.dev/v1alpha1` and every earlier field with its meaning + +### Requirement: Error document +A command that fails SHALL exit 1. With `--json` it SHALL print a document of kind `Error` +with a stable `code` and a human-readable `message`; without `--json` it SHALL print +`error: ` on stderr, followed by a hint line when there is one. + +#### Scenario: An unknown flag with --json +- **WHEN** `siding version --json --nope` runs +- **THEN** it exits 1 and prints an `Error` document with `code` `InvalidArguments`, even though flag parsing failed before `--json` was read + +#### Scenario: A failure without --json +- **WHEN** `siding up` runs outside a git repository +- **THEN** it exits 1 and prints `error: …` and a hint on stderr, and nothing on stdout + +#### Scenario: The error codes +- **WHEN** a command fails +- **THEN** its code is one of `InvalidArguments`, `NotAGitRepository`, `NoBranch`, `ComposeFileNotFound`, `ComposeFailed`, `DockerUnavailable`, `ProxyNotRunning`, `PortInUse`, `InvalidRoute`, `AmbiguousPort`, `SlugCollision`, `StackNotFound`, `UnmergedBranch` or `DirtyWorktree` + +### Requirement: Exit codes +siding SHALL exit 0 on success and when a stack is Ready, 1 when a command fails or a stack is +Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`), and 3 for +NoCapacity, which v0.1 never returns. + +#### Scenario: A verdict decides the exit code +- **WHEN** `siding status --wait` ends with the verdict Failed +- **THEN** it exits 1, and with Ready it exits 0 + +### Requirement: Version +`siding version` SHALL print the version, the commit (when the build knows it), the Go version +and the platform; with `--json` as a document of kind `Version` with the fields `version`, +`commit`, `goVersion` and `platform`. + +#### Scenario: Text output +- **WHEN** `siding version` runs in a build of commit `0123456789abcdef…` +- **THEN** it prints one line such as `siding v0.1.0 (0123456789ab) go1.27.3 linux/arm64` + +#### Scenario: An argument +- **WHEN** `siding version extra` runs +- **THEN** it fails with `InvalidArguments` diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/dev-environment/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/dev-environment/spec.md new file mode 100644 index 0000000..f96a988 --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/specs/dev-environment/spec.md @@ -0,0 +1,21 @@ +# Spec Delta + +## ADDED Requirements + +### Requirement: Stack smoke test +`just smoke-stacks` SHALL run siding's own commands end to end against the inner daemon: +`setup`, `up --wait`, routing by host and path, a `Failed` verdict with log tail, `ps`, `down` +and the refusal of `down --volumes`. Its proxy, network and projects MUST carry the prefix +`siding-test-`, and its proxy MUST listen on 18080. + +#### Scenario: Stack smoke test in CI +- **WHEN** a pull request is opened or pushed to, or a commit is pushed to `main` +- **THEN** CI runs `just smoke-stacks` inside the devcontainer, against its own daemon + +#### Scenario: It leaves nothing behind +- **WHEN** `just smoke-stacks` ends, passing or failing +- **THEN** no container, network or volume whose name starts with `siding-test-` is left in the inner daemon, and `siding-proxy` and the network `siding` are untouched + +#### Scenario: Not part of level 1 +- **WHEN** `just test` runs without a Docker daemon +- **THEN** it passes, because the end-to-end tests only build with the `docker` build tag diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/proxy/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/proxy/spec.md new file mode 100644 index 0000000..f95cad9 --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/specs/proxy/spec.md @@ -0,0 +1,67 @@ +# Spec Delta + +## Purpose + +Defines the one shared proxy through which every stack is reached: `siding-proxy`, a Traefik +on the network `siding`, started by `siding setup` and bound to the loopback interface only. + +## ADDED Requirements + +### Requirement: Setup starts the proxy +`siding setup` SHALL create the network `siding` if it is missing and run the Compose project +`siding` with one container `siding-proxy`: Traefik with the Docker provider on the network +`siding`, restarting unless stopped. It SHALL print the proxy's address, with `--json` as a +`Setup` document. + +#### Scenario: First setup +- **WHEN** `siding setup` runs on a machine without the proxy +- **THEN** the network `siding` and the running container `siding-proxy` exist, and it prints `http://*.localhost` as the address + +#### Scenario: JSON +- **WHEN** `siding setup --json` runs +- **THEN** it prints a `Setup` document with the proxy's container name, network, image and port + +### Requirement: Loopback port +The proxy SHALL publish exactly one port, on `127.0.0.1` only, `80` unless `--port ` names +another. siding MUST read the proxy's port from the running container, not from a file. + +#### Scenario: Default port +- **WHEN** `siding setup` has run +- **THEN** `siding-proxy` publishes `127.0.0.1:80` and no other port, and nothing listens for it on other interfaces + +#### Scenario: Another port +- **WHEN** `siding setup --port 8080` has run and `siding open` runs later +- **THEN** the address it opens is `http://:8080` + +#### Scenario: The port is taken +- **WHEN** another process holds `127.0.0.1:80` and `siding setup` runs +- **THEN** it fails with `PortInUse` and a hint to use `--port` + +### Requirement: Setup is idempotent +`siding setup` SHALL change nothing when the proxy already runs as requested, and SHALL +recreate only the proxy container when its port or image differs. Running stacks MUST keep +running and be routed again once the proxy is back. + +#### Scenario: Second run +- **WHEN** `siding setup` runs twice +- **THEN** the second run reports the proxy as already running and recreates nothing + +#### Scenario: Port change with running stacks +- **WHEN** a stack runs and `siding setup --port 8080` follows `siding setup` +- **THEN** the stack's containers are not restarted and it answers on port 8080 + +### Requirement: Only siding stacks are routed +The proxy MUST route only containers that carry `dev.siding.repo` and enable Traefik, so that +other containers with Traefik labels on the same daemon are never exposed through it. + +#### Scenario: A foreign Traefik label +- **WHEN** a container without siding labels carries `traefik.enable=true` and a Host rule +- **THEN** the proxy answers 404 for that host + +### Requirement: Stacks need the proxy +`up` and `status` SHALL fail with `ProxyNotRunning` and the hint `siding setup` when +`siding-proxy` is not running. + +#### Scenario: up before setup +- **WHEN** `siding up` runs on a machine without the proxy +- **THEN** it fails with `ProxyNotRunning` before starting any container diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/routing/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/routing/spec.md new file mode 100644 index 0000000..645e64c --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/specs/routing/spec.md @@ -0,0 +1,89 @@ +# Spec Delta + +## Purpose + +Defines how the services of a stack are reached through the proxy: which services count as +HTTP services, how the label `dev.siding.route` maps them to the stack's host and paths, and +how `siding open` finds the address. + +## ADDED Requirements + +### Requirement: HTTP services +A service SHALL be an HTTP service when the user's compose file gives it exactly one distinct +container port, from the targets of `ports` and from `expose`, or when its label +`dev.siding.port` names one. A service with several ports and no `dev.siding.port` is not one. + +#### Scenario: One published port +- **WHEN** a service publishes `"3000:3000"` +- **THEN** it is an HTTP service on container port 3000, and the port is no longer published + +#### Scenario: Several ports with the label +- **WHEN** a service exposes 3000 and 9229 and carries `dev.siding.port: "3000"` +- **THEN** it is an HTTP service on container port 3000 + +#### Scenario: No port +- **WHEN** a worker service has neither `ports` nor `expose` nor `dev.siding.port` +- **THEN** it is not routed + +### Requirement: Service host +Every HTTP service SHALL answer at `...localhost`, unless it +carries `dev.siding.route: "false"`. + +#### Scenario: An unlabelled HTTP service +- **WHEN** the service `mail` exposes 8025 and has no route label, in the stack `shop` on branch `main` +- **THEN** `mail.shop.main.localhost` reaches it on 8025 + +#### Scenario: Hidden +- **WHEN** a service carries `dev.siding.route: "false"` +- **THEN** no host or path reaches it through the proxy + +### Requirement: Route label +`dev.siding.route: "/"` SHALL make an HTTP service the root of the stack's host, and +`dev.siding.route: "/"` SHALL mount it on the stack's host under that path prefix, +passing the path on unchanged. The longest matching prefix MUST win. + +#### Scenario: Root and API +- **WHEN** `web` carries `dev.siding.route: "/"` and `api` carries `dev.siding.route: "/api"` in the stack `shop` on `main` +- **THEN** `shop.main.localhost/` reaches `web`, and `shop.main.localhost/api/users` reaches `api` with the path `/api/users` + +#### Scenario: No root +- **WHEN** no service carries `dev.siding.route: "/"` +- **THEN** a path on the stack's host that no route covers answers 404 + +### Requirement: Invalid routes +`siding up` SHALL fail with `InvalidRoute`, naming the service, and start nothing when a route +label is neither `false` nor an absolute path, when two services claim the same path, or when +a service with a route is not an HTTP service. + +#### Scenario: Two services on one path +- **WHEN** `web` and `admin` both carry `dev.siding.route: "/"` +- **THEN** `siding up` fails with `InvalidRoute` naming both + +#### Scenario: A route without a port +- **WHEN** a service carries `dev.siding.route: "/api"` but has no container port +- **THEN** `siding up` fails with `InvalidRoute` and a hint to add `expose` or `dev.siding.port` + +#### Scenario: A route with several ports +- **WHEN** a service carries `dev.siding.route: "/"` and exposes 3000 and 9229 without `dev.siding.port` +- **THEN** `siding up` fails with `AmbiguousPort` naming the ports + +### Requirement: Services that cannot join the proxy network +A service with a `network_mode` SHALL NOT be routed; `up` SHALL warn about it on stderr, and +fail with `InvalidRoute` when it carries a route path. + +#### Scenario: network_mode host +- **WHEN** a service uses `network_mode: host` and has no route label +- **THEN** `siding up` starts it unrouted and prints a warning + +### Requirement: Open +`siding open [service]` SHALL open the stack's address, or the service's host, in the default +browser; with `--print` or `--json` it SHALL only print it, as text or as an `Address` +document. The address MUST include the proxy's port unless it is 80. + +#### Scenario: Print the address +- **WHEN** the proxy listens on 8080 and `siding open --print` runs in the stack `shop` on `main` +- **THEN** it prints `http://shop.main.localhost:8080/` + +#### Scenario: Unknown service +- **WHEN** `siding open nope` runs and the stack has no HTTP service `nope` +- **THEN** it fails with `InvalidArguments` and lists the HTTP services diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/stack-identity/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/stack-identity/spec.md new file mode 100644 index 0000000..816eb25 --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/specs/stack-identity/spec.md @@ -0,0 +1,101 @@ +# Spec Delta + +## Purpose + +Defines how siding derives a stack's identity from a git worktree: repository name, branch +slug, Compose project, host names, machine id and the labels that let Docker answer every +later question about the stack. + +## ADDED Requirements + +### Requirement: One stack per worktree +siding SHALL treat the git worktree containing the current directory as the stack's worktree, +identified by its absolute top-level path. Every command that acts on a stack MUST act on that +worktree's stack. + +#### Scenario: Run from a subdirectory +- **WHEN** `siding status` runs in `/src/shop-login/web/src`, inside the worktree `/src/shop-login` +- **THEN** it reports the stack of `/src/shop-login` + +#### Scenario: Outside a repository +- **WHEN** a stack command runs outside any git worktree +- **THEN** it fails with `NotAGitRepository` + +### Requirement: Repository name +The repository name SHALL be the name of the directory that holds the repository's common git +directory (the main worktree), the same for every worktree of the repository. `--repo ` +MUST override it. + +#### Scenario: Two worktrees of one repository +- **WHEN** `/src/shop` is the main worktree and `/src/shop-login` a linked worktree of it +- **THEN** both have the repository name `shop` + +#### Scenario: Override +- **WHEN** `siding up --repo store` runs in `/src/shop` +- **THEN** the repository name is `store` + +### Requirement: Slugs +The repository name and the branch SHALL each be turned into a slug: lowercase, every run of +characters other than `a-z` and `0-9` replaced by one `-`, no leading or trailing `-`. A slug +longer than 40 characters MUST be cut to 33 characters, followed by `-` and the first 6 hex +digits of the SHA-256 of the original name. + +#### Scenario: A branch with a slash +- **WHEN** the branch is `feature/Login_Form` +- **THEN** its slug is `feature-login-form` + +#### Scenario: A long branch +- **WHEN** the branch name is 60 characters long +- **THEN** its slug is 40 characters long and ends in `-` and 6 hex digits that differ for another branch with the same first 33 characters + +### Requirement: Branch required +The worktree SHALL be on a branch. In a detached HEAD a stack command MUST fail with `NoBranch` +and a hint to check out a branch. + +#### Scenario: Detached HEAD +- **WHEN** `siding up` runs in a worktree whose HEAD is detached +- **THEN** it fails with `NoBranch` and starts nothing + +### Requirement: Project name and host +The Compose project SHALL be named `-` and the stack's host SHALL be +`..localhost`. + +#### Scenario: Names +- **WHEN** the repository is `shop` and the branch `feature/login` +- **THEN** the project is `shop-feature-login` and the host `shop.feature-login.localhost` + +### Requirement: Slug collision +siding MUST NOT let two worktrees or branches share one project. If the project already exists +with another `dev.siding.worktree` or `dev.siding.branch` label, `up` MUST fail with +`SlugCollision`, naming the other worktree and branch, and change nothing. + +#### Scenario: feature/a and feature-a +- **WHEN** a stack for branch `feature/a` runs and `siding up` runs in a worktree on branch `feature-a` +- **THEN** it fails with `SlugCollision` and the running stack is untouched + +### Requirement: Labels +siding SHALL label every container, network and volume of a stack with `dev.siding.repo`, +`dev.siding.branch` (the branch name, not the slug), `dev.siding.worktree` (absolute path) and +`dev.siding.machine`, and with `dev.siding.agent` when `--agent` or `SIDING_AGENT` names one. +External networks and volumes MUST NOT be relabelled. + +#### Scenario: Labels on a running stack +- **WHEN** `siding up` has run in `/src/shop-login` on branch `feature/login` +- **THEN** every container, the project's networks and its named volumes carry `dev.siding.repo=shop`, `dev.siding.branch=feature/login` and `dev.siding.worktree=/src/shop-login` + +#### Scenario: An agent name +- **WHEN** `SIDING_AGENT=claude siding up` runs +- **THEN** the containers also carry `dev.siding.agent=claude` + +### Requirement: Machine id +`dev.siding.machine` SHALL be a stable id of the machine siding runs on: derived from the +operating system's machine id by a one-way hash, never the raw id. Without an operating system +id, siding MUST generate one and keep it in `~/.siding/machine-id`. + +#### Scenario: Two runs +- **WHEN** `siding up` runs twice on the same machine +- **THEN** both stacks carry the same `dev.siding.machine` + +#### Scenario: No operating system id +- **WHEN** neither `/etc/machine-id` nor the macOS platform UUID is available +- **THEN** siding writes a random id to `~/.siding/machine-id` and uses it from then on diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md new file mode 100644 index 0000000..a742ffd --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md @@ -0,0 +1,125 @@ +# Spec Delta + +## Purpose + +Defines how a worktree's stack is started, inspected through its logs and taken down: the +user's compose file stays untouched, siding adds one override, and nothing removes unmerged or +uncommitted work without being forced. + +## ADDED Requirements + +### Requirement: Compose file +siding SHALL use the compose file Compose itself would pick in the worktree's top-level +directory (`compose.yaml`, `compose.yml`, `docker-compose.yaml`, `docker-compose.yml`), or the +one named by `--file`. Without one, stack commands MUST fail with `ComposeFileNotFound`. + +#### Scenario: The default file +- **WHEN** the worktree holds `compose.yaml` +- **THEN** `siding up` uses it + +#### Scenario: No compose file +- **WHEN** the worktree holds no compose file and `--file` is not given +- **THEN** `siding up` fails with `ComposeFileNotFound` + +### Requirement: The compose file is never edited +siding MUST NOT write to the user's compose file, its `.env` or any other file of the user. It +SHALL write only `.siding/override.yml` and `.siding/.gitignore` (containing `*`) in the +worktree, so that `.siding/` never makes the worktree dirty. + +#### Scenario: After up +- **WHEN** `siding up` has run in a clean worktree +- **THEN** `git status --porcelain` prints nothing and the compose file is byte for byte unchanged + +### Requirement: Override +The override SHALL reset every service's `ports` (`ports: !reset []`), add every routed +service to the network `siding` next to its own networks, and add the `dev.siding.*` and +Traefik labels. The user's labels and settings MUST otherwise stay in effect. + +#### Scenario: A published port +- **WHEN** the compose file publishes `"5432:5432"` for `db` +- **THEN** after `siding up` no container of the stack publishes a port + +#### Scenario: Own networks +- **WHEN** `api` is on the user's networks `backend` and `default` +- **THEN** after `siding up` it is on `backend`, `default` and `siding` + +### Requirement: Up +`siding up` SHALL run `docker compose -p -f -f .siding/override.yml +up -d --remove-orphans`, followed by any arguments after `--`. It SHALL print the stack's +address, with `--json` as a `Stack` document, and fail with `ComposeFailed` and Compose's +message when Compose fails. + +#### Scenario: Passing arguments through +- **WHEN** `siding up -- --build web` runs +- **THEN** Compose builds and starts `web` and its dependencies + +#### Scenario: Compose fails +- **WHEN** an image of the compose file does not exist +- **THEN** `siding up` exits 1 with `ComposeFailed` and Compose's error in the message + +#### Scenario: Run twice +- **WHEN** `siding up` runs again in the same worktree +- **THEN** the same project is updated, and no second stack appears + +### Requirement: Up and wait +`siding up --wait [--timeout ]` SHALL start the stack as `up` does and then end exactly like +`siding status --wait`, with its document and exit code. + +#### Scenario: An agent starts and waits +- **WHEN** an agent runs `siding up --wait --json` and every service becomes ready +- **THEN** it prints one `Status` document with the verdict Ready and exits 0 + +### Requirement: Down +`siding down` SHALL remove the stack's containers and networks by project name, also when the +compose file is gone, and keep its volumes. It SHALL print what it removed, with `--json` as a +`Down` document, and with `--dry-run` remove nothing. + +#### Scenario: Data survives +- **WHEN** `siding down` and then `siding up` run +- **THEN** the stack's named volumes still hold their data + +#### Scenario: Dry run +- **WHEN** `siding down --dry-run --json` runs +- **THEN** it prints a `Down` document listing the containers and networks with `dryRun: true`, and everything still runs + +#### Scenario: Nothing running +- **WHEN** `siding down` runs in a worktree without a stack +- **THEN** it fails with `StackNotFound` + +### Requirement: Removing data keeps work +`siding down --volumes` SHALL also remove the stack's volumes, but MUST refuse with +`UnmergedBranch` when the branch is not merged into the default branch, and with +`DirtyWorktree` when `git status` lists changes, unless `--force` is given. + +#### Scenario: Unmerged branch +- **WHEN** `siding down --volumes` runs on a branch with commits that the default branch lacks +- **THEN** it fails with `UnmergedBranch`, removes nothing, and the hint names `--force` + +#### Scenario: Dirty worktree +- **WHEN** the worktree has uncommitted changes and `siding down --volumes` runs +- **THEN** it fails with `DirtyWorktree` and removes nothing + +#### Scenario: Forced +- **WHEN** `siding down --volumes --force` runs on an unmerged branch +- **THEN** the containers, networks and volumes of the stack are removed + +#### Scenario: No default branch +- **WHEN** siding finds neither `origin/HEAD` nor a local `main` or `master` +- **THEN** the branch counts as unmerged + +### Requirement: Logs +`siding logs [service…]` SHALL show the logs of the stack's services, following them with +`-f` and limited with `--tail `. With `--json` it SHALL print a `Logs` document of the last +lines per service; `--json` together with `-f` MUST fail with `InvalidArguments`. + +#### Scenario: One service +- **WHEN** `siding logs api --tail 5` runs +- **THEN** it prints the last 5 lines of `api` + +### Requirement: No .env values in JSON +Log lines in any document SHALL have every value defined in the worktree's `.env` and in the +services' `env_file`s, of 4 or more characters, replaced by `***`. + +#### Scenario: A password in a log line +- **WHEN** `.env` sets `DB_PASSWORD=not-a-real-token` and a service logs that value +- **THEN** the log line in the `Logs` and `Status` documents shows `***` instead diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/stack-list/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/stack-list/spec.md new file mode 100644 index 0000000..df633c4 --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/specs/stack-list/spec.md @@ -0,0 +1,51 @@ +# Spec Delta + +## Purpose + +Defines `siding ps`, the one view of every stack on the machine: what runs, for which branch +and worktree, at which address, and what memory it takes right now. + +## ADDED Requirements + +### Requirement: Every stack on the machine +`siding ps` SHALL list every stack whose containers carry `dev.siding.repo`, found through the +Docker API from any directory, with repository, branch, state, address, memory in use and +worktree. It needs no git repository and no file of siding's. + +#### Scenario: Two worktrees +- **WHEN** stacks run for `shop` on `main` and on `feature/login` +- **THEN** `siding ps` lists both, each with its own address + +#### Scenario: A stack whose worktree is gone +- **WHEN** a worktree has been deleted while its stack still runs +- **THEN** `siding ps` still lists the stack, and marks the worktree as missing + +#### Scenario: Containers of other projects +- **WHEN** the daemon also runs containers without siding labels +- **THEN** `siding ps` does not list them + +### Requirement: Stack state in the list +The state in `ps` SHALL be `running` when every container of the stack runs, `partial` when +some do, and `stopped` when none does. `ps` MUST NOT probe the proxy; the full verdict is +`siding status`. + +#### Scenario: A stopped container +- **WHEN** one of three containers of a stack has exited +- **THEN** `siding ps` shows the stack as `partial` + +### Requirement: Memory in use +`ps` SHALL show each stack's memory in use, the sum over its running containers as the Docker +API reports it, and the total over all stacks. + +#### Scenario: Two stacks +- **WHEN** one stack uses 300 MiB and another 500 MiB +- **THEN** `siding ps` shows 300 MiB and 500 MiB for them and 800 MiB in total + +### Requirement: StackList document +With `--json`, `ps` SHALL print one `StackList` document with one entry per stack: project, +repository, branch, worktree, machine, agent, state, address, memory in bytes, and +`worktreeMissing` when the worktree is gone. + +#### Scenario: No stacks +- **WHEN** `siding ps --json` runs and no stack exists +- **THEN** it prints a `StackList` document without stacks and exits 0 diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/stack-status/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/stack-status/spec.md new file mode 100644 index 0000000..ee85bed --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/specs/stack-status/spec.md @@ -0,0 +1,85 @@ +# Spec Delta + +## Purpose + +Defines the verdict on a stack that agents and humans wait for: Ready only when it actually +runs and answers through the proxy, otherwise the service that failed, the reason, a hint and +the last lines of its log. + +## ADDED Requirements + +### Requirement: Verdict +`siding status` SHALL report one verdict for the worktree's stack: `Ready`, `Starting`, +`Failed` or `TimedOut`, the state of every service, and the stack's address. Without `--wait` +it SHALL report the current state once. + +#### Scenario: Status without wait +- **WHEN** `siding status` runs while a healthcheck is still starting +- **THEN** it reports `Starting` and exits 2 + +#### Scenario: No stack +- **WHEN** `siding status` runs in a worktree whose project has no containers +- **THEN** it fails with `StackNotFound` and the hint `siding up` + +### Requirement: Ready +A stack SHALL be `Ready` only when every service of the compose file has a container, every +service with a healthcheck is healthy, every other service is running or has exited 0, and +every service with a route path answers through the proxy. + +#### Scenario: Healthy and answering +- **WHEN** `db` is healthy, `web` with `dev.siding.route: "/"` runs and answers through the proxy, and `migrate` has exited 0 +- **THEN** the verdict is `Ready` + +#### Scenario: Running but not answering +- **WHEN** `web` with `dev.siding.route: "/"` runs but its process does not listen yet +- **THEN** the verdict is `Starting`, not `Ready` + +### Requirement: Answering through the proxy +A routed service SHALL count as answering when a request to its route through the proxy gets +an HTTP response from the service itself, of any status; a response the proxy produces on its +own (no router yet, 502, 503, 504) MUST NOT count. + +#### Scenario: The app answers 404 +- **WHEN** `api` with `dev.siding.route: "/api"` answers `/api` with 404 +- **THEN** `api` counts as answering + +#### Scenario: Bad gateway +- **WHEN** the proxy answers 502 for `/api` because nothing listens on the container port +- **THEN** `api` does not count as answering + +### Requirement: Failed +A stack SHALL be `Failed` as soon as a service has exited with a non-zero code, is unhealthy, +keeps restarting, or has no container; the failure MUST name the service, a reason +(`Exited`, `Unhealthy`, `Restarting`, `Missing`, `NotAnswering`), a hint, and the last 20 log +lines of that service. + +#### Scenario: A service exits +- **WHEN** `api` exits with code 1 after printing `cannot connect to db` +- **THEN** the verdict is `Failed` with service `api`, reason `Exited`, the exit code in the hint, and the log tail containing `cannot connect to db` + +#### Scenario: Unhealthy +- **WHEN** `db`'s healthcheck reports unhealthy +- **THEN** the verdict is `Failed` with reason `Unhealthy`, and the hint contains the last healthcheck output + +### Requirement: Waiting +`siding status --wait` SHALL check until the verdict is `Ready` or `Failed`, at most for +`--timeout` (default 3 minutes). When the time is up it SHALL report `TimedOut`, naming the +first service that is not ready, with reason, hint and log tail; a routed service that never +answered gets the reason `NotAnswering`. + +#### Scenario: Ready in time +- **WHEN** `siding status --wait --timeout 60s` runs and the stack becomes ready after 20 seconds +- **THEN** it reports `Ready` after about 20 seconds and exits 0 + +#### Scenario: Too slow +- **WHEN** `siding status --wait --timeout 10s` runs and `web` with route `/` never answers +- **THEN** it reports `TimedOut` with service `web`, reason `NotAnswering`, and exits 2 + +### Requirement: Status document +With `--json`, `status` SHALL print one `Status` document: the verdict, the stack (project, +repository, branch, worktree, address), every service (state, health, route, address, +answering), and for `Failed` and `TimedOut` the failure with service, reason, hint and log tail. + +#### Scenario: An agent reads the failure +- **WHEN** `siding status --wait --json` ends with `Failed` +- **THEN** the document holds `verdict: Failed` and a `failure` with `service`, `reason`, `hint` and `logs`, and the process exits 1 diff --git a/openspec/changes/phase-1-stacks-and-routing/tasks.md b/openspec/changes/phase-1-stacks-and-routing/tasks.md new file mode 100644 index 0000000..9119ba5 --- /dev/null +++ b/openspec/changes/phase-1-stacks-and-routing/tasks.md @@ -0,0 +1,71 @@ +# Tasks + +## 1. Foundations + +- [ ] 1.1 Add `Git`, `Compose`, `Docker`, `Probe`, `Browser`, `Clock`, `Home`, `Env` and `Proxy` to `cli.Options` as interfaces or values (D14), with real implementations wired in `cmd/siding/main.go` and fakes in a test helper; verify `just test` passes and no package-level variable is used as a seam +- [ ] 1.2 Add every error code of the spec `cli-contract` with a hint field on `Error`, printed as a second stderr line in text mode and as `hint` in the `Error` document; verify with a table test per code and the updated `error.json` golden +- [ ] 1.3 Extend the exit codes with 2 (TimedOut/Starting) and 3 (NoCapacity, unused) and give `Execute` a way to return a verdict's exit code; verify with a unit test of `Execute` + +## 2. Stack identity + +- [ ] 2.1 Implement `internal/git`: top level, common dir, current branch (error on detached HEAD), dirty check, merged-into-default-branch check; verify with table tests against temporary repositories (`git init` in `t.TempDir()`), including linked worktrees, detached HEAD, a missing default branch and `.siding/` being ignored +- [ ] 2.2 Implement slugs, project name and hosts in `internal/stack`; verify with a table test covering slashes, case, runs of symbols, leading and trailing symbols, 40/41/60 characters and the hash suffix +- [ ] 2.3 Implement the machine id (D6) with the os id source and `~/.siding/machine-id` from `Options.Home`; verify that two calls agree, that the raw id never appears, and that the fallback file is created once +- [ ] 2.4 Implement the label set (`dev.siding.repo`, `branch`, `worktree`, `machine`, optional `agent` from `--agent` or `SIDING_AGENT`); verify with a unit test + +## 3. Compose config, routing and the override + +- [ ] 3.1 Implement `internal/compose`: find the compose file (`--file` or Compose's order), run `config --format json`, parse services, ports, `expose`, labels, networks, `network_mode`, healthchecks, named and external volumes and networks, `env_file`; check Compose >= 2.24; verify with golden inputs captured from the devcontainer's Compose in `internal/compose/testdata/` +- [ ] 3.2 Implement HTTP detection and the routing table (service hosts, route paths, `false`, `dev.siding.port`) with the `InvalidRoute` and `AmbiguousPort` errors and the `network_mode` warning; verify with a table test per scenario of the spec `routing` +- [ ] 3.3 Generate `.siding/override.yml` (D7) including `ports: !reset []`, networks, Traefik routers, the `X-Siding-Service` headers middleware and labels on own networks and volumes; verify with golden files in `internal/stack/testdata/override/` (`just golden`) and by checking that `docker compose config` accepts each golden override in the devcontainer +- [ ] 3.4 Add `dev.siding.port` to the label conventions in CLAUDE.md and `openspec/config.yaml`; verify `openspec validate --all --strict` passes + +## 4. Docker API and the proxy + +- [ ] 4.1 Add `github.com/moby/moby/client` and implement `internal/docker`: containers and their state by project label, all siding stacks, published ports of a container, logs tail, one-shot memory stats, network and volume lists by label; `DockerUnavailable` when the daemon is unreachable; verify with unit tests of the mapping from API types to siding's types +- [ ] 4.2 Implement `internal/proxy` and `siding setup [--port] [--json]` (D4): embedded compose file, network `siding`, `127.0.0.1:`, the `dev.siding.repo` constraint, idempotence, recreate on port or image change, `PortInUse`; verify with fake-based tests of the decisions and a `setup.json` golden +- [ ] 4.3 Document `siding setup` in the README (port 80, `--port`, `--port 8080` in the devcontainer, `*.localhost` resolution); verify the commands in the README match `siding setup --help` + +## 5. Up + +- [ ] 5.1 Implement `siding up [--file] [--repo] [--agent] [--json] [-- args]`: identity, `ProxyNotRunning`, `SlugCollision`, write `.siding/override.yml` and `.siding/.gitignore`, run Compose with `--remove-orphans` and the passed-through arguments, `ComposeFailed`, Compose progress on stderr; verify with fake-based tests per scenario of the spec `stack-lifecycle` (Up, Override, never edited) and a `stack.json` golden +- [ ] 5.2 Document `siding up` in the README, including that published ports are reset and how to reach a database (`docker compose exec`); verify the example runs as written in the devcontainer + +## 6. Verdict and status + +- [ ] 6.1 Implement the probe (D9) against `Options.Probe`: `Host` header, no redirects, `X-Siding-Service` and the 502/503/504 rule; verify with an `httptest` server for each case +- [ ] 6.2 Implement `internal/verdict` (D10): Ready, Starting, Failed with `Exited`, `Unhealthy`, `Restarting`, `Missing`, TimedOut with `NotAnswering`, hints, order of evaluation, first failing service in compose order; verify with a table test per scenario of the spec `stack-status` +- [ ] 6.3 Implement `siding status [--wait] [--timeout] [--json]` with polling through `Options.Clock`, the 20-line log tail, `StackNotFound`, exit codes 0/1/2; verify with fake-based tests including a timeout without real waiting, and `status-ready.json`, `status-failed.json` and `status-timedout.json` goldens +- [ ] 6.4 Add `siding up --wait [--timeout]` ending like `status --wait`; verify with a fake-based test that one `Status` document is printed and the exit code follows the verdict + +## 7. ps + +- [ ] 7.1 Implement `siding ps [--json]`: all stacks by `dev.siding.repo`, state `running`/`partial`/`stopped`, address with the proxy port, memory per stack and in total, `worktreeMissing`; verify with fake-based tests and `stacklist.json` and `stacklist-empty.json` goldens +- [ ] 7.2 Document `siding ps` and `siding status` in the README; verify the shown output matches the text golden of the tests + +## 8. Logs, open and down + +- [ ] 8.1 Implement redaction (D12): parse `.env` and `env_file`s, replace values of 4+ characters, longest first; verify with a table test using `not-a-real-token` and quoted, commented and short values +- [ ] 8.2 Implement `siding logs [service…] [-f] [--tail] [--json]` (Compose for text, the Docker API and redaction for the `Logs` document, `InvalidArguments` for `--json -f`); apply redaction to the status log tail too; verify with fake-based tests and a `logs.json` golden +- [ ] 8.3 Implement `siding open [service] [--print] [--json]` with `Options.Browser` (`open` on macOS, `$BROWSER` or `xdg-open` on Linux) and the `Address` document; verify with fake-based tests for port 80, 8080 and an unknown service, and an `address.json` golden +- [ ] 8.4 Implement `siding down [--volumes] [--force] [--dry-run] [--json]` (D11) by project name, the `UnmergedBranch` and `DirtyWorktree` guard for `--volumes`, `StackNotFound`, the `Down` document; verify with fake-based tests per scenario and `down.json` and `down-dryrun.json` goldens +- [ ] 8.5 Make the guardrail "never lose work" precise in CLAUDE.md and `openspec/config.yaml` (data and worktrees are guarded, containers are not) and document `logs`, `open` and `down` in the README; verify `just test` passes + +## 9. Agent skill + +- [ ] 9.1 Write the skill text (`internal/cli/skill/SKILL.md`) and `siding skill [--json]` with the `Skill` document; verify with a golden of the printed text, a test that it names `siding up --wait --json`, `status --json`, `logs --json` and exit codes 0/1/2 and none of `--force`, `--volumes`, `share`, and a `skill.json` golden +- [ ] 9.2 Document installing the skill in the README; verify the shown command produces the file in a temporary directory + +## 10. End-to-end and CI + +- [ ] 10.1 Write `internal/e2e` with `//go:build docker` (D14): proxy `siding-test-proxy` on network `siding-test` at 18080, temporary repositories with `--repo siding-test-…`, a two-service app with `/` and `/api`, cleanup by exact name in `t.Cleanup`; cases: setup idempotent, `up --wait` Ready, Host and path routing, the service host, `X-Siding-Service` present on app responses and the 502 rule, a failing service giving `Failed` with log tail and a redacted `.env` value, `ps`, `down` keeping volumes, `down --volumes` refused when dirty and unmerged and allowed with `--force`; verify with `just smoke-stacks` in the devcontainer +- [ ] 10.2 Add the recipe `smoke-stacks` (`hack/smoke/stacks.sh` with `require_devcontainer`, running `go test -tags docker ./internal/e2e`) to the justfile and the CI smoke job, and verify afterwards that no `siding-test-*` object is left in the inner daemon and that `just test` still passes without Docker +- [ ] 10.3 Update the dev-environment table in CLAUDE.md and the README with `just smoke-stacks` and `siding setup --port 8080` for the devcontainer; verify the commands run as written +- [ ] 10.4 Run the whole flow by hand in the devcontainer: `just build`, `bin/siding setup --port 8080`, `up --wait` in a sample project, open `http://.localhost:8080` from the host browser, `ps`, `down`; record the result in the phase summary +- [ ] 10.5 Update the status in CLAUDE.md, the README's pre-alpha note and the status in `openspec/config.yaml` to v0.1; verify `just test` passes + +## Workflow follow-up + +- Stop and show the result; wait for the maintainer's approval of Phase 1. +- `/opsx:archive` merges the deltas into `openspec/specs/`. +- Tag `v0.1.0` after archive, if the maintainer wants a release. From 0aaa5141711feb041e2efc0a9920e2cb20282642 Mon Sep 17 00:00:00 2001 From: tweinmann Date: Wed, 28 Oct 2026 20:45:21 +0000 Subject: [PATCH 2/3] Implement phase 1: stacks and routing (v0.1) siding setup, up [--wait], status [--wait], ps, logs, open, down [--volumes], skill, and routing by host and path through siding-proxy. Every command has --json; exit codes 0 Ready, 1 Failed, 2 TimedOut. Docker state comes from the Engine API (moby client), lifecycle from docker compose with the user's file plus .siding/override.yml. Unit tests run against fakes and golden files; just smoke-stacks runs the commands end to end against the devcontainer's inner daemon. The devcontainer now requires the host's port 8080 instead of letting VS Code forward the proxy to another port. Co-Authored-By: Claude Opus 5.5 --- .devcontainer/devcontainer.json | 7 +- .github/workflows/ci.yml | 14 +- .golangci.yml | 4 + CLAUDE.md | 36 +- README.md | 83 +++- cmd/siding/main.go | 39 +- go.mod | 24 +- go.sum | 61 +++ hack/smoke/stacks.sh | 27 ++ internal/cli/cli.go | 162 +++++++- internal/cli/cli_test.go | 86 +++- internal/cli/down.go | 159 ++++++++ internal/cli/down_test.go | 118 ++++++ internal/cli/env_test.go | 373 ++++++++++++++++++ internal/cli/json.go | 138 ++++++- internal/cli/logs.go | 94 +++++ internal/cli/logs_test.go | 62 +++ internal/cli/open.go | 85 ++++ internal/cli/open_test.go | 68 ++++ internal/cli/ps.go | 144 +++++++ internal/cli/ps_test.go | 70 ++++ internal/cli/setup.go | 120 ++++++ internal/cli/setup_test.go | 85 ++++ internal/cli/skill.go | 36 ++ internal/cli/skill/SKILL.md | 51 +++ internal/cli/skill_test.go | 45 +++ internal/cli/stack.go | 171 ++++++++ internal/cli/status.go | 229 +++++++++++ internal/cli/status_test.go | 175 ++++++++ internal/cli/testdata/json/address.json | 6 + internal/cli/testdata/json/down-dryrun.json | 22 ++ internal/cli/testdata/json/down.json | 18 + internal/cli/testdata/json/logs.json | 27 ++ internal/cli/testdata/json/setup.json | 10 + internal/cli/testdata/json/skill.json | 6 + internal/cli/testdata/json/stack.json | 30 ++ .../cli/testdata/json/stacklist-empty.json | 4 + internal/cli/testdata/json/stacklist.json | 29 ++ internal/cli/testdata/json/status-failed.json | 51 +++ internal/cli/testdata/json/status-ready.json | 42 ++ .../cli/testdata/json/status-timedout.json | 46 +++ internal/cli/testdata/skill/SKILL.md | 51 +++ internal/cli/up.go | 173 ++++++++ internal/cli/up_test.go | 163 ++++++++ internal/compose/compose.go | 165 ++++++++ internal/compose/config.go | 188 +++++++++ internal/compose/config_test.go | 126 ++++++ internal/compose/testdata/routing/.env | 1 + internal/compose/testdata/routing/api.env | 1 + .../compose/testdata/routing/compose.yaml | 40 ++ internal/compose/testdata/routing/config.json | 156 ++++++++ internal/docker/docker.go | 324 +++++++++++++++ internal/docker/docker_test.go | 64 +++ internal/e2e/e2e_test.go | 309 +++++++++++++++ internal/fail/fail.go | 60 +++ internal/git/git.go | 147 +++++++ internal/git/git_test.go | 158 ++++++++ internal/probe/probe.go | 53 +++ internal/probe/probe_test.go | 55 +++ internal/proxy/proxy.go | 66 ++++ internal/stack/identity.go | 111 ++++++ internal/stack/identity_test.go | 126 ++++++ internal/stack/machineid.go | 68 ++++ internal/stack/override.go | 117 ++++++ internal/stack/redact.go | 90 +++++ internal/stack/redact_test.go | 38 ++ internal/stack/routing.go | 116 ++++++ internal/stack/routing_test.go | 193 +++++++++ internal/stack/testdata/override/routing.yml | 134 +++++++ internal/verdict/verdict.go | 191 +++++++++ internal/verdict/verdict_test.go | 144 +++++++ justfile | 7 +- .../phase-1-stacks-and-routing/design.md | 33 +- .../specs/cli-contract/spec.md | 10 +- .../specs/dev-environment/spec.md | 20 + .../specs/stack-identity/spec.md | 4 +- .../specs/stack-status/spec.md | 4 +- .../phase-1-stacks-and-routing/tasks.md | 69 ++-- openspec/config.yaml | 17 +- 79 files changed, 6748 insertions(+), 101 deletions(-) create mode 100755 hack/smoke/stacks.sh create mode 100644 internal/cli/down.go create mode 100644 internal/cli/down_test.go create mode 100644 internal/cli/env_test.go create mode 100644 internal/cli/logs.go create mode 100644 internal/cli/logs_test.go create mode 100644 internal/cli/open.go create mode 100644 internal/cli/open_test.go create mode 100644 internal/cli/ps.go create mode 100644 internal/cli/ps_test.go create mode 100644 internal/cli/setup.go create mode 100644 internal/cli/setup_test.go create mode 100644 internal/cli/skill.go create mode 100644 internal/cli/skill/SKILL.md create mode 100644 internal/cli/skill_test.go create mode 100644 internal/cli/stack.go create mode 100644 internal/cli/status.go create mode 100644 internal/cli/status_test.go create mode 100644 internal/cli/testdata/json/address.json create mode 100644 internal/cli/testdata/json/down-dryrun.json create mode 100644 internal/cli/testdata/json/down.json create mode 100644 internal/cli/testdata/json/logs.json create mode 100644 internal/cli/testdata/json/setup.json create mode 100644 internal/cli/testdata/json/skill.json create mode 100644 internal/cli/testdata/json/stack.json create mode 100644 internal/cli/testdata/json/stacklist-empty.json create mode 100644 internal/cli/testdata/json/stacklist.json create mode 100644 internal/cli/testdata/json/status-failed.json create mode 100644 internal/cli/testdata/json/status-ready.json create mode 100644 internal/cli/testdata/json/status-timedout.json create mode 100644 internal/cli/testdata/skill/SKILL.md create mode 100644 internal/cli/up.go create mode 100644 internal/cli/up_test.go create mode 100644 internal/compose/compose.go create mode 100644 internal/compose/config.go create mode 100644 internal/compose/config_test.go create mode 100644 internal/compose/testdata/routing/.env create mode 100644 internal/compose/testdata/routing/api.env create mode 100644 internal/compose/testdata/routing/compose.yaml create mode 100644 internal/compose/testdata/routing/config.json create mode 100644 internal/docker/docker.go create mode 100644 internal/docker/docker_test.go create mode 100644 internal/e2e/e2e_test.go create mode 100644 internal/fail/fail.go create mode 100644 internal/git/git.go create mode 100644 internal/git/git_test.go create mode 100644 internal/probe/probe.go create mode 100644 internal/probe/probe_test.go create mode 100644 internal/proxy/proxy.go create mode 100644 internal/stack/identity.go create mode 100644 internal/stack/identity_test.go create mode 100644 internal/stack/machineid.go create mode 100644 internal/stack/override.go create mode 100644 internal/stack/redact.go create mode 100644 internal/stack/redact_test.go create mode 100644 internal/stack/routing.go create mode 100644 internal/stack/routing_test.go create mode 100644 internal/stack/testdata/override/routing.yml create mode 100644 internal/verdict/verdict.go create mode 100644 internal/verdict/verdict_test.go diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index f64d77b..ac98417 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -39,10 +39,13 @@ ], // The proxy (Traefik) listens on 8080 of the inner daemon; this forwards it to the host, so - // http://.localhost:8080 reaches a stack from the browser. + // http://.localhost:8080 reaches a stack from the browser. requireLocalPort: when 8080 + // is taken on the host, VS Code says so instead of forwarding to another port, which would + // make every address siding prints wrong. To use another port, change both numbers here and + // run siding setup --port with it. "forwardPorts": [8080], "portsAttributes": { - "8080": { "label": "siding proxy", "onAutoForward": "silent" } + "8080": { "label": "siding proxy", "onAutoForward": "silent", "requireLocalPort": true } }, // Worktrees with relative paths work both inside the container and on the host. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 067e518..3806cea 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,8 +1,8 @@ -# Level-1 checks and the Docker smoke test. Both run inside the devcontainer image, so CI uses -# exactly the tool versions pinned in .devcontainer/Dockerfile, and the smoke test runs against +# Level-1 checks and the Docker smoke tests. All run inside the devcontainer image, so CI uses +# exactly the tool versions pinned in .devcontainer/Dockerfile, and the smoke tests run against # the devcontainer's own Docker daemon, as locally. # -# `just test` runs on every push. The smoke test runs on pull requests and on main. +# `just test` runs on every push. The smoke tests run on pull requests and on main. name: ci on: @@ -39,14 +39,16 @@ jobs: export GOCACHE="$PWD/.cache/go/build" GOMODCACHE="$PWD/.cache/go/mod" just test - smoke-docker: + smoke: if: github.event_name == 'pull_request' || github.ref == 'refs/heads/main' runs-on: ubuntu-24.04-arm timeout-minutes: 30 steps: - uses: actions/checkout@v7 - - name: just smoke-docker (in the devcontainer) + - name: just smoke-docker and smoke-stacks (in the devcontainer) uses: devcontainers/ci@v0.3 with: push: never - runCmd: just smoke-docker + runCmd: | + just smoke-docker + just smoke-stacks diff --git a/.golangci.yml b/.golangci.yml index 6e4047f..fe09c66 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -1,5 +1,9 @@ version: "2" +# Also lint the end-to-end tests, which only build with the tag docker (just smoke-stacks). +run: + build-tags: [docker] + linters: default: standard enable: diff --git a/CLAUDE.md b/CLAUDE.md index d1df133..94de012 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -14,10 +14,11 @@ changing architecture. ## Status -Phase 0 done, waiting for approval (2026-10-28): devcontainer with Docker-in-Docker, OpenSpec -with the spec `dev-environment` and the draft changes `phase-1-…` to `phase-5-…` (v0.1–v0.5), -`siding version [--json]`, `just test`, `just smoke-docker`, `hack/nuke.sh`, CI. -Next: Phase 1 (`phase-1-stacks-and-routing`, v0.1). +Phase 0 approved (2026-10-28). Phase 1 (`phase-1-stacks-and-routing`, v0.1) implemented, +waiting for approval (2026-10-28), not yet archived: `setup`, `up [--wait]`, `status [--wait]`, +`ps`, `logs`, `open`, `down [--volumes]`, `skill`, routing by host and path through +`siding-proxy`, `just smoke-stacks`. After approval: `/opsx:archive`, then Phase 2 +(`phase-2-overview-and-budget`, v0.2). ## Working agreements @@ -52,8 +53,10 @@ Next: Phase 1 (`phase-1-stacks-and-routing`, v0.1). the last log lines of that service. - Agent and human see the same thing. Every reading command has `--json`; exit codes are 0 Ready, 1 Failed, 2 TimedOut, 3 NoCapacity. -- Never lose work. Removing a stack or worktree refuses when the branch is unmerged or the - worktree is dirty, unless forced; every destructive command supports `--dry-run`. +- Never lose work. Removing a stack's data (its volumes) or a worktree refuses when the branch + is unmerged or the worktree is dirty, unless forced; removing containers and networks loses + nothing and is not guarded (`siding down` keeps volumes, `down --volumes` is guarded). Every + destructive command supports `--dry-run`. - Only a human exposes anything to the internet. `siding share` is interactive only and has no `--yes`. - Standard building blocks (Docker Compose, Traefik, later Sablier) over custom code. @@ -66,11 +69,14 @@ Next: Phase 1 (`phase-1-stacks-and-routing`, v0.1). All development happens inside the devcontainer. It runs its own Docker daemon (Docker-in-Docker), which also hosts every stack siding's tests create. Nothing is installed on the host. Tool versions, Compose included, are pinned in `.devcontainer/Dockerfile`, the Docker engine in -`.devcontainer/devcontainer.json`. The proxy port 8080 is forwarded to the host, so -`http://.localhost:8080` reaches a stack from the browser. +`.devcontainer/devcontainer.json`. The proxy port 8080 is forwarded to the host: run +`bin/siding setup --port 8080` once in the container, and `http://.localhost:8080` +reaches a stack from the browser. The forward requires the host's 8080 (`requireLocalPort`): +when it is taken, VS Code says so instead of moving to another port. Free it, or change the +port in `devcontainer.json` and use the same one with `siding setup --port`. -CI runs `just test` on every push and `just smoke-docker` on pull requests and on `main`, both -inside the devcontainer image. +CI runs `just test` on every push and `just smoke-docker` and `just smoke-stacks` on pull +requests and on `main`, all inside the devcontainer image. | Command | Run from | Purpose | |---|---|---| @@ -78,6 +84,8 @@ inside the devcontainer image. | `just golden` | devcontainer | rewrite golden files after an intended change | | `just build` | devcontainer | build a static `bin/siding` (linux) | | `just smoke-docker` | devcontainer | inner daemon: compose project with two services and healthchecks, `ports: !reset []`, Traefik on `siding` routing `demo.main.localhost` at `/` and `/api` | +| `just smoke-stacks` | devcontainer | inner daemon: siding's own commands end to end (`internal/e2e`, build tag `docker`): setup, `up --wait`, routing, a Failed verdict with log tail, `ps`, `down`, the guard of `down --volumes`; also checks golden overrides with `docker compose config` | +| `bin/siding setup --port 8080` | devcontainer | the real `siding-proxy` on the forwarded port, to try siding by hand | | `openspec validate --all --strict` | devcontainer | check specs and active changes (part of `just test`) | | `openspec list` / `openspec list --specs` / `openspec view` | devcontainer | active changes, specs, dashboard | | `/opsx:explore`, `/opsx:propose`, `/opsx:apply`, `/opsx:archive` | Claude Code | the change workflow; skills in `.claude/skills/`, regenerated by `openspec update` | @@ -94,7 +102,9 @@ inside the devcontainer image. with `prune`. New host objects go into that spec, `hack/nuke.sh` and (for volumes) `devcontainer.json` together, in one change. - Tests create stacks only as Compose projects with the prefix `siding-test-` and remove them by - their exact name. The smoke test's proxy listens on 18080, never on 8080. + their exact name. The smoke tests' proxies listen on 18080, never on 8080; the end-to-end + tests run their own proxy `siding-test-e2e-proxy` on the network `siding-test-e2e`, so a + developer's `siding-proxy` is never touched. - No real secrets in the repo, in golden files or in logs; golden files may contain obvious fakes such as `not-a-real-token`. `.env` values never appear in `--json` output. - A worktree created with relative paths sets `extensions.relativeWorktrees` in `.git/config`, @@ -108,7 +118,9 @@ inside the devcontainer image. - Module `github.com/tweinmann/siding`, binary `siding`, API version `siding.dev/v1alpha1`. - Labels on every container, network and volume siding creates or adopts: `dev.siding.repo`, `dev.siding.branch`, `dev.siding.worktree` (absolute path), `dev.siding.machine` (stable - machine id), `dev.siding.agent` (optional). Service labels: `dev.siding.route`, + machine id); on containers also `dev.siding.agent` (optional; never on networks or volumes, + whose labels must not change between two `up`s). Service labels: `dev.siding.route` + (`/`, `/`, `false`), `dev.siding.port` (the HTTP port of a service with several), `dev.siding.seed` (`volume|script|none`). Volume label `dev.siding.shared`. - Compose project `-`, host `..localhost`, proxy `siding-proxy` on the network `siding`, override `.siding/override.yml`. diff --git a/README.md b/README.md index 8be9165..e2280f0 100644 --- a/README.md +++ b/README.md @@ -3,9 +3,78 @@ siding gives every git worktree its own Docker Compose stack, at its own `*.localhost` address, and tells you and your coding agents whether it actually runs and what it costs. -> **Pre-alpha.** Nothing here is usable yet: `siding version` is the only command. The plan is -> in [openspec/changes/](openspec/changes) (one change per phase), what siding does today in -> [openspec/specs/](openspec/specs). +> **Pre-alpha (v0.1).** Stacks per worktree, routing through one local proxy and the verdict +> of `siding status` work; expect rough edges and changes before v1. There are no releases +> yet: build it from source (see below). The plan is in [openspec/changes/](openspec/changes) +> (one change per phase), what siding does today in [openspec/specs/](openspec/specs). + +## Using siding + +siding needs Docker with Compose 2.24 or later, and git. Your compose file stays as it is. + +### Once per machine: the proxy + +```sh +siding setup # Traefik as siding-proxy on 127.0.0.1:80 +siding setup --port 8080 # if port 80 is taken, or inside siding's devcontainer +``` + +Every stack is reached through this one proxy at `http://..localhost`. Browsers +and curl resolve `*.localhost` to your machine by themselves; for other tools add the host to +`/etc/hosts`. Running `siding setup` again changes nothing unless the port differs. + +### In a worktree: start the stack + +```sh +siding up # e.g. shop on feature/login: http://shop.feature-login.localhost/ +siding up -- --build web # arguments after -- go to docker compose up +siding up --wait # and wait for the verdict, like siding status --wait +``` + +`up` runs your compose file as the Compose project `-` with one more file, +`.siding/override.yml`. That file resets every published port, so branches never fight over +ports, and puts siding's labels and routes on the services. `.siding/` ignores itself in git. + +- A service labelled `dev.siding.route: "/"` is the stack's root, `dev.siding.route: "/api"` + answers under that path (the path is passed on unchanged), `dev.siding.route: "false"` + hides a service. +- Every service with one container port (from `ports` or `expose`) also answers at + `http://...localhost`. With several ports, `dev.siding.port: "3000"` + names the HTTP one. +- Published ports are gone, also a database's. Reach it with + `docker compose -p - exec db psql …`. + +### Does it run? + +```sh +siding status # Ready, Starting, Failed or TimedOut, per service +siding status --wait # until Ready or Failed, at most --timeout (3m) +siding ps # every stack on this machine: state, address, memory +``` + +`Ready` means every service with a healthcheck is healthy and every service with a route +answers through the proxy. A failure names the service, the reason, a hint and its last log +lines. Exit codes: 0 Ready, 1 Failed, 2 TimedOut or still starting. Every command prints one +JSON document with `--json`, for scripts and agents. + +### Logs, address, and taking it down + +```sh +siding logs -f api # docker compose logs of the stack +siding open # the stack's address in the browser; siding open api for a service +siding down # remove containers and networks; volumes and data stay +siding down --volumes # also the data; refused while the branch is unmerged or the + # worktree dirty, unless --force; --dry-run lists it first +``` + +### For coding agents + +`siding skill` prints a Claude Code skill that tells an agent to run `siding up --wait --json` +after a change and to act on the verdict. siding writes no file itself; install it with + +```sh +mkdir -p .claude/skills/siding && siding skill > .claude/skills/siding/SKILL.md +``` ## Developing siding @@ -22,6 +91,14 @@ on your machine and siding's tests never touch your Docker containers. | `just golden` | rewrite golden files after an intended output change | | `just build` | build `bin/siding` | | `just smoke-docker` | Compose, `ports: !reset []` and Traefik routing against the container's own daemon | + | `just smoke-stacks` | siding's own commands end to end against the container's own daemon | + + To try siding by hand: `just build`, then `bin/siding setup --port 8080` (the port the + devcontainer forwards), and `bin/siding up --wait` in a project; its address, + `http://..localhost:8080`, opens in the browser on your machine. Port 8080 + must be free on your machine: VS Code reports it when it is not, rather than forwarding to + another port. Then free it, or change the port in `.devcontainer/devcontainer.json` and pass + the same one to `siding setup --port`. To remove everything siding development left on your Docker host, run `hack/nuke.sh` in a terminal on the host (`--dry-run` lists it first). diff --git a/cmd/siding/main.go b/cmd/siding/main.go index fcd5d16..778f220 100644 --- a/cmd/siding/main.go +++ b/cmd/siding/main.go @@ -3,14 +3,51 @@ package main import ( + "context" + "fmt" "os" + "os/signal" + "syscall" "github.com/tweinmann/siding/internal/cli" + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/probe" + "github.com/tweinmann/siding/internal/proxy" + "github.com/tweinmann/siding/internal/stack" ) // version is set at build time with -ldflags "-X main.version=". var version = "dev" func main() { - os.Exit(cli.Execute(cli.New(cli.Options{Build: cli.CurrentBuild(version)}), os.Args[1:], os.Stdout, os.Stderr)) + dir, err := os.Getwd() + if err != nil { + fmt.Fprintf(os.Stderr, "error: %v\n", err) + os.Exit(1) + } + home, err := os.UserHomeDir() + if err != nil { + fmt.Fprintf(os.Stderr, "error: %v\n", err) + os.Exit(1) + } + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) + opts := cli.Options{ + Build: cli.CurrentBuild(version), + Dir: dir, + Home: home, + Getenv: os.Getenv, + OSMachineID: stack.OSMachineID, + Git: git.Client{}, + Compose: &compose.CLI{}, + Docker: &docker.Client{}, + Probe: probe.HTTP{Header: stack.ServiceHeader}, + Browser: cli.SystemBrowser{Getenv: os.Getenv}, + Clock: cli.SystemClock{}, + Proxy: proxy.Default(), + } + code := cli.Execute(ctx, cli.New(opts), os.Args[1:], os.Stdout, os.Stderr) + stop() + os.Exit(code) } diff --git a/go.mod b/go.mod index 8e69a95..fb4aef7 100644 --- a/go.mod +++ b/go.mod @@ -2,9 +2,31 @@ module github.com/tweinmann/siding go 1.27.3 -require github.com/spf13/cobra v1.10.2 +require ( + github.com/containerd/errdefs v1.0.0 + github.com/moby/moby/api v1.56.1 + github.com/moby/moby/client v0.6.2 + github.com/spf13/cobra v1.10.2 +) require ( + github.com/Microsoft/go-winio v0.6.2 // indirect + github.com/containerd/errdefs/pkg v0.3.0 // indirect + github.com/distribution/reference v0.6.0 // indirect + github.com/docker/go-connections v0.8.2 // indirect + github.com/docker/go-units v0.5.0 // indirect + github.com/felixge/httpsnoop v1.0.4 // indirect + github.com/go-logr/logr v1.4.2 // indirect + github.com/go-logr/stdr v1.2.2 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/moby/docker-image-spec v1.3.1 // indirect + github.com/opencontainers/go-digest v1.0.0 // indirect + github.com/opencontainers/image-spec v1.1.1 // indirect github.com/spf13/pflag v1.0.9 // indirect + go.opentelemetry.io/auto/sdk v1.1.0 // indirect + go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 // indirect + go.opentelemetry.io/otel v1.35.0 // indirect + go.opentelemetry.io/otel/metric v1.35.0 // indirect + go.opentelemetry.io/otel/trace v1.35.0 // indirect + golang.org/x/sys v0.33.0 // indirect ) diff --git a/go.sum b/go.sum index a6ee3e0..e23e7ca 100644 --- a/go.sum +++ b/go.sum @@ -1,10 +1,71 @@ +github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= +github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= +github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI= +github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M= +github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE= +github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk= github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= +github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= +github.com/docker/go-connections v0.8.2 h1:9d9DEokx5W7qlr0shkFXIxEA7+TTD+llwzcykSlxMNc= +github.com/docker/go-connections v0.8.2/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q= +github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= +github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= +github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= +github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= +github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY= +github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0= +github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo= +github.com/moby/moby/api v1.56.1 h1:PpWkvVPB7Fr/No8w+TfyJ/I6rWZ2YPZhT3JorIg+06c= +github.com/moby/moby/api v1.56.1/go.mod h1:sZ+THbVWkjOmBPPfbnzdD/G1LuIexWhqlSHHPTDQ1Uk= +github.com/moby/moby/client v0.6.2 h1:ekSPlMccrZGcw6Scb8T9D39qE0mB+XupU/yS0zNw9N4= +github.com/moby/moby/client v0.6.2/go.mod h1:DGcnO7qmPJrqobx/ggQgQ/AUk+2wgVdQv7uxB0/mEsc= +github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= +github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= +github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040= +github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU= github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4= github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY= github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA= +go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 h1:sbiXRNDSWJOTobXh5HyQKjq6wUC5tNybqjIqDpAY4CU= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0/go.mod h1:69uWxva0WgAA/4bu2Yy70SLDBwZXuQ6PbBpbsa5iZrQ= +go.opentelemetry.io/otel v1.35.0 h1:xKWKPxrxB6OtMCbmMY021CqC45J+3Onta9MqjhnusiQ= +go.opentelemetry.io/otel v1.35.0/go.mod h1:UEqy8Zp11hpkUrL73gSlELM0DupHoiq72dR+Zqel/+Y= +go.opentelemetry.io/otel/metric v1.35.0 h1:0znxYu2SNyuMSQT4Y9WDWej0VpcsxkuklLa4/siN90M= +go.opentelemetry.io/otel/metric v1.35.0/go.mod h1:nKVFgxBZ2fReX6IlyW28MgZojkoAkJGaE8CpgeAU3oE= +go.opentelemetry.io/otel/sdk v1.35.0 h1:iPctf8iprVySXSKJffSS79eOjl9pvxV9ZqOWT0QejKY= +go.opentelemetry.io/otel/sdk v1.35.0/go.mod h1:+ga1bZliga3DxJ3CQGg3updiaAJoNECOgJREo9KHGQg= +go.opentelemetry.io/otel/sdk/metric v1.35.0 h1:1RriWBmCKgkeHEhM7a2uMjMUfP7MsOF5JpUCaEqEI9o= +go.opentelemetry.io/otel/sdk/metric v1.35.0/go.mod h1:is6XYCUMpcKi+ZsOvfluY5YstFnhW0BidkR+gL+qN+w= +go.opentelemetry.io/otel/trace v1.35.0 h1:dPpEfJu1sDIqruz7BHFG3c7528f6ddfSWfFDVt/xgMs= +go.opentelemetry.io/otel/trace v1.35.0/go.mod h1:WUk7DtFp1Aw2MkvqGdwiXYDZZNvA/1J8o6xRXLrIkyc= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= +golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw= +golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q= +gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA= +pgregory.net/rapid v1.2.0 h1:keKAYRcjm+e1F0oAuU5F5+YPAWcyxNNRK2wud503Gnk= +pgregory.net/rapid v1.2.0/go.mod h1:PY5XlDGj0+V1FCq0o192FdRhpKHGTRIWBgqjDBTrq04= diff --git a/hack/smoke/stacks.sh b/hack/smoke/stacks.sh new file mode 100755 index 0000000..5e3ed37 --- /dev/null +++ b/hack/smoke/stacks.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# End-to-end test of siding's own commands against the devcontainer's inner daemon: setup, +# up --wait, routing by host and path, a Failed verdict with its log tail, ps, down and the +# refusal of down --volumes. It runs the Go tests with the build tag docker: +# - internal/stack with -compose: docker compose accepts every golden override; +# - internal/e2e: the commands themselves, with the proxy siding-test-e2e-proxy on the +# network siding-test-e2e at port 18080 and projects named siding-test-*. +# Every object is removed by its exact name, also when a test fails. +set -euo pipefail +source "$(dirname "$0")/../lib.sh" +require_devcontainer +require_compose 2.24 + +cd "$(dirname "$0")/../.." +go test -count=1 ./internal/stack -compose +go test -count=1 -tags docker ./internal/e2e + +# Prove that nothing is left behind. +left="$( + docker ps -a --format '{{.Names}}' + docker network ls --format '{{.Name}}' + docker volume ls --format '{{.Name}}' +)" +if grep -q "^${SIDING_TEST_PREFIX}" <<<"$left"; then + die "objects named ${SIDING_TEST_PREFIX}* are left in the inner daemon: $(grep "^${SIDING_TEST_PREFIX}" <<<"$left" | tr '\n' ' ')" +fi +echo "smoke-stacks: ok" diff --git a/internal/cli/cli.go b/internal/cli/cli.go index 45ff1dd..4a33737 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -3,28 +3,103 @@ package cli import ( + "context" "errors" "fmt" "io" + "time" "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/probe" + "github.com/tweinmann/siding/internal/proxy" ) // Exit codes. They are part of the contract with agents, like the --json documents: 0 Ready, -// 1 Failed, 2 TimedOut, 3 NoCapacity. A command that fails with an Error document exits 1. +// 1 Failed, 2 TimedOut (or Starting without --wait), 3 NoCapacity. A command that fails with +// an Error document exits 1. const ( - exitOK = 0 - exitFailed = 1 + exitOK = 0 + exitFailed = 1 + exitTimedOut = 2 + // exitNoCapacity is reserved for the memory budget (v0.2). + exitNoCapacity = 3 + // exitInterrupted ends a command stopped by Ctrl-C or SIGTERM, without a message, as + // shells do. + exitInterrupted = 130 ) // Options holds everything a command needs from the machine it runs on. Tests pass their own, // so no package-level variable is a test seam and tests can run in parallel. type Options struct { Build BuildInfo + // Dir is the current directory. + Dir string + // Home is the user's home directory, for the cache under ~/.siding. + Home string + // Getenv reads an environment variable. + Getenv func(string) string + // OSMachineID returns the operating system's machine id, or "". + OSMachineID func() string + + Git Git + Compose Compose + Docker Docker + Probe Prober + Browser Browser + Clock Clock + // Proxy names siding-proxy and its network. + Proxy proxy.Config +} + +// Git answers questions about worktrees. +type Git interface { + Worktree(ctx context.Context, dir string) (git.Worktree, error) + Dirty(ctx context.Context, top string) (bool, error) + Merged(ctx context.Context, top, branch string) (merged bool, base string, err error) +} + +// Compose runs docker compose. +type Compose interface { + Config(ctx context.Context, inv compose.Invocation) ([]byte, error) + Run(ctx context.Context, inv compose.Invocation, args []string, stdout, stderr io.Writer) error +} + +// Docker reads state from the Docker Engine API. +type Docker interface { + Ping(ctx context.Context) error + Containers(ctx context.Context, labels map[string]string) ([]docker.Container, error) + Container(ctx context.Context, name string) (*docker.Container, error) + Networks(ctx context.Context, labels map[string]string) ([]docker.Network, error) + Volumes(ctx context.Context, labels map[string]string) ([]docker.Volume, error) + EnsureNetwork(ctx context.Context, name string, labels map[string]string) (bool, error) + Logs(ctx context.Context, id string, tail int) ([]string, error) + Memory(ctx context.Context, id string) (uint64, error) +} + +// Prober sends one request through the proxy. +type Prober interface { + Get(ctx context.Context, addr, host, path string) (probe.Response, error) +} + +// Browser opens a URL. +type Browser interface { + Open(url string) error +} + +// Clock is the time, for waiting. +type Clock interface { + Now() time.Time + Sleep(ctx context.Context, d time.Duration) error } // New returns the root command. func New(opts Options) *cobra.Command { + a := &app{opts: opts} root := &cobra.Command{ Use: "siding", Short: "One Docker Compose stack per git worktree, and one view of all of them", @@ -32,51 +107,84 @@ func New(opts Options) *cobra.Command { SilenceErrors: true, } root.CompletionOptions.DisableDefaultCmd = true + // An error is one line (and a hint); cobra's suggestions would add several. + root.DisableSuggestions = true root.SetFlagErrorFunc(func(_ *cobra.Command, err error) error { return &Error{Code: CodeInvalidArguments, Message: err.Error()} }) - root.AddCommand(newVersionCommand(opts.Build)) + root.AddCommand( + newVersionCommand(opts.Build), + newSetupCommand(a), + newUpCommand(a), + newStatusCommand(a), + newPsCommand(a), + newLogsCommand(a), + newOpenCommand(a), + newDownCommand(a), + newSkillCommand(), + ) return root } +// app gives commands the Options. +type app struct { + opts Options +} + // Execute runs root with args and returns the exit code. A failure is printed as an Error -// document on stdout when --json was given, and as one line on stderr otherwise. -func Execute(root *cobra.Command, args []string, stdout, stderr io.Writer) int { +// document on stdout when --json was given, and as one line (and a hint) on stderr otherwise. +func Execute(ctx context.Context, root *cobra.Command, args []string, stdout, stderr io.Writer) int { root.SetArgs(args) root.SetOut(stdout) root.SetErr(stderr) - cmd, err := root.ExecuteC() + cmd, err := root.ExecuteContextC(ctx) if err == nil { return exitOK } + var code exitCode + if errors.As(err, &code) { + return int(code) + } + if ctx.Err() != nil && errors.Is(err, ctx.Err()) { + return exitInterrupted + } var e *Error if !errors.As(err, &e) { - // Cobra reports an unknown command as a plain error. + // Cobra reports an unknown command as a plain error; anything else unexpected is + // still reported in the same shape. e = &Error{Code: CodeInvalidArguments, Message: err.Error()} + if cmd != nil && cmd != root { + e.Code = CodeInternal + } } if wantsJSON(cmd, args) { - if perr := printJSON(stdout, errorDocument{header: newHeader(kindError), Code: e.Code, Message: e.Message}); perr != nil { + doc := errorDocument{header: newHeader(kindError), Code: e.Code, Message: e.Message, Hint: e.Hint} + if perr := printJSON(stdout, doc); perr != nil { _, _ = fmt.Fprintf(stderr, "error: %s\n", e.Message) } } else { _, _ = fmt.Fprintf(stderr, "error: %s\n", e.Message) + if e.Hint != "" { + _, _ = fmt.Fprintf(stderr, "hint: %s\n", e.Hint) + } } return exitFailed } -// Error is a failure with a stable code, which agents may branch on. Codes are never renamed -// within an apiVersion. -type Error struct { - Code string - Message string -} +// exitCode ends a command that has already printed its result with this exit code. +type exitCode int -func (e *Error) Error() string { return e.Message } +func (c exitCode) Error() string { return fmt.Sprintf("exit code %d", int(c)) } -// Error codes. +// Error is a failure with a stable code, which agents may branch on. +type Error = fail.Error + +// Error codes Execute uses itself; the rest are in package fail. const ( // CodeInvalidArguments: unknown command, flag or argument. - CodeInvalidArguments = "InvalidArguments" + CodeInvalidArguments = fail.InvalidArguments + // CodeInternal: an unexpected error without a code of its own. + CodeInternal = fail.Internal ) // noArgs refuses positional arguments with CodeInvalidArguments. @@ -86,3 +194,21 @@ func noArgs(cmd *cobra.Command, args []string) error { } return nil } + +// SystemClock is the real time. +type SystemClock struct{} + +// Now returns the current time. +func (SystemClock) Now() time.Time { return time.Now() } + +// Sleep waits for d or until ctx is done. +func (SystemClock) Sleep(ctx context.Context, d time.Duration) error { + t := time.NewTimer(d) + defer t.Stop() + select { + case <-ctx.Done(): + return ctx.Err() + case <-t.C: + return nil + } +} diff --git a/internal/cli/cli_test.go b/internal/cli/cli_test.go index 9636e9e..a202261 100644 --- a/internal/cli/cli_test.go +++ b/internal/cli/cli_test.go @@ -2,9 +2,15 @@ package cli import ( "bytes" + "context" + "errors" "path/filepath" "testing" + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/testutil" ) @@ -17,7 +23,7 @@ var testBuild = BuildInfo{ func run(args ...string) (code int, stdout, stderr string) { var out, errOut bytes.Buffer - code = Execute(New(Options{Build: testBuild}), args, &out, &errOut) + code = Execute(context.Background(), New(Options{Build: testBuild}), args, &out, &errOut) return code, out.String(), errOut.String() } @@ -98,3 +104,81 @@ func TestErrorDocumentOnFlagError(t *testing.T) { t.Errorf("stdout:\n%s\nwant:\n%s", stdout, want) } } + +// Every error code reaches the user the same way: one line and a hint on stderr, or an Error +// document with code, message and hint on stdout. +func TestErrorCodes(t *testing.T) { + t.Parallel() + codes := []string{ + fail.InvalidArguments, fail.NotAGitRepository, fail.NoBranch, fail.ComposeFileNotFound, + fail.ComposeFailed, fail.DockerUnavailable, fail.ProxyNotRunning, fail.PortInUse, + fail.InvalidRoute, fail.AmbiguousPort, fail.SlugCollision, fail.StackNotFound, + fail.UnmergedBranch, fail.DirtyWorktree, fail.Internal, + } + for _, code := range codes { + t.Run(code, func(t *testing.T) { + t.Parallel() + root := &cobra.Command{Use: "siding", SilenceUsage: true, SilenceErrors: true} + var asJSON bool + sub := &cobra.Command{Use: "boom", RunE: func(*cobra.Command, []string) error { + return fail.New(code, "it broke").WithHint("try again") + }} + registerJSON(sub, &asJSON) + root.AddCommand(sub) + + var out, errOut bytes.Buffer + if got := Execute(context.Background(), root, []string{"boom"}, &out, &errOut); got != exitFailed { + t.Errorf("exit code %d", got) + } + if out.String() != "" || errOut.String() != "error: it broke\nhint: try again\n" { + t.Errorf("text: stdout %q, stderr %q", out.String(), errOut.String()) + } + + out.Reset() + errOut.Reset() + root.SetArgs(nil) + Execute(context.Background(), root, []string{"boom", "--json"}, &out, &errOut) + want := "{\n \"apiVersion\": \"siding.dev/v1alpha1\",\n \"kind\": \"Error\",\n \"code\": \"" + code + + "\",\n \"message\": \"it broke\",\n \"hint\": \"try again\"\n}\n" + if out.String() != want || errOut.String() != "" { + t.Errorf("json: stdout %s, stderr %q", out.String(), errOut.String()) + } + }) + } +} + +// A command that printed a verdict ends with its exit code and nothing more. +func TestExitCodes(t *testing.T) { + t.Parallel() + for _, code := range []int{exitFailed, exitTimedOut, exitNoCapacity} { + root := &cobra.Command{Use: "siding", SilenceUsage: true, SilenceErrors: true} + root.AddCommand(&cobra.Command{Use: "verdict", RunE: func(*cobra.Command, []string) error { return exitCode(code) }}) + var out, errOut bytes.Buffer + if got := Execute(context.Background(), root, []string{"verdict"}, &out, &errOut); got != code || out.Len()+errOut.Len() != 0 { + t.Errorf("exit code %d (want %d), output %q %q", got, code, out.String(), errOut.String()) + } + } + // An error without a code is still reported in the same shape. + root := &cobra.Command{Use: "siding", SilenceUsage: true, SilenceErrors: true} + root.AddCommand(&cobra.Command{Use: "oops", RunE: func(*cobra.Command, []string) error { return errors.New("disk full") }}) + var out, errOut bytes.Buffer + if got := Execute(context.Background(), root, []string{"oops"}, &out, &errOut); got != exitFailed || errOut.String() != "error: disk full\n" { + t.Errorf("exit code %d, stderr %q", got, errOut.String()) + } +} + +// Ctrl-C ends a command quietly with 130, e.g. logs --follow. +func TestInterrupted(t *testing.T) { + t.Parallel() + ctx, cancel := context.WithCancel(context.Background()) + root := &cobra.Command{Use: "siding", SilenceUsage: true, SilenceErrors: true} + root.AddCommand(&cobra.Command{Use: "follow", RunE: func(cmd *cobra.Command, _ []string) error { + cancel() + <-cmd.Context().Done() + return cmd.Context().Err() + }}) + var out, errOut bytes.Buffer + if got := Execute(ctx, root, []string{"follow"}, &out, &errOut); got != exitInterrupted || out.Len()+errOut.Len() != 0 { + t.Errorf("exit code %d, output %q %q", got, out.String(), errOut.String()) + } +} diff --git a/internal/cli/down.go b/internal/cli/down.go new file mode 100644 index 0000000..484e245 --- /dev/null +++ b/internal/cli/down.go @@ -0,0 +1,159 @@ +package cli + +import ( + "context" + "fmt" + "sort" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" +) + +type downFlags struct { + volumes, force, dryRun bool +} + +func newDownCommand(a *app) *cobra.Command { + var asJSON bool + var f downFlags + cmd := &cobra.Command{ + Use: "down", + Short: "Remove the worktree's stack; its volumes stay unless --volumes", + Long: `Remove the containers and networks of the worktree's stack. Its volumes, and with them +its data, stay. --volumes removes them too, but refuses while the branch is not merged into the +default branch or the worktree has uncommitted changes, unless --force.`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + doc, err := a.down(cmd, f) + if err != nil { + return err + } + if asJSON { + return printJSON(cmd.OutOrStdout(), doc) + } + return printDownText(cmd, doc, f.volumes) + }, + } + cmd.Flags().BoolVar(&f.volumes, "volumes", false, "also remove the stack's volumes and their data") + cmd.Flags().BoolVar(&f.force, "force", false, "remove the volumes even if the branch is unmerged or the worktree dirty") + cmd.Flags().BoolVar(&f.dryRun, "dry-run", false, "list what would be removed, remove nothing") + registerJSON(cmd, &asJSON) + return cmd +} + +func (a *app) down(cmd *cobra.Command, f downFlags) (downDocument, error) { + ctx := cmd.Context() + st, err := a.findStack(ctx) + if err != nil { + return downDocument{}, err + } + if f.volumes && !f.force { + if err := a.keepsWork(ctx, st); err != nil { + return downDocument{}, err + } + } + doc, err := a.downPlan(ctx, st, f) + if err != nil { + return downDocument{}, err + } + if f.dryRun { + return doc, nil + } + args := []string{"down", "--remove-orphans"} + if f.volumes { + args = append(args, "--volumes") + } + // By project name, without files: it works even when the compose file is gone. + inv := compose.Invocation{Dir: st.wt.Top, Project: st.project} + if err := a.opts.Compose.Run(ctx, inv, args, cmd.ErrOrStderr(), cmd.ErrOrStderr()); err != nil { + return downDocument{}, err + } + return doc, nil +} + +// keepsWork refuses to remove data while the worktree has changes or the branch is unmerged. +func (a *app) keepsWork(ctx context.Context, st existing) error { + dirty, err := a.opts.Git.Dirty(ctx, st.wt.Top) + if err != nil { + return err + } + if dirty { + return fail.New(fail.DirtyWorktree, "the worktree %s has uncommitted changes; its volumes stay", st.wt.Top). + WithHint("commit or stash them first, or remove the data anyway with --force") + } + merged, base, err := a.opts.Git.Merged(ctx, st.wt.Top, st.wt.Branch) + if err != nil { + return err + } + if !merged { + into := "no default branch was found (origin/HEAD, main or master)" + if base != "" { + into = "it is not merged into " + strings.TrimPrefix(strings.TrimPrefix(base, "refs/remotes/"), "refs/heads/") + } + return fail.New(fail.UnmergedBranch, "the branch %s may hold unmerged work: %s; its volumes stay", st.wt.Branch, into). + WithHint("merge it first, or remove the data anyway with --force (a squash merge also needs --force)") + } + return nil +} + +// downPlan lists what down removes. +func (a *app) downPlan(ctx context.Context, st existing, f downFlags) (downDocument, error) { + doc := downDocument{header: newHeader(kindDown), Stack: st.ref(a.shownPort(ctx)), DryRun: f.dryRun} + byProject := map[string]string{stack.LabelProject: st.project} + cs, err := a.opts.Docker.Containers(ctx, byProject) + if err != nil { + return doc, err + } + for _, c := range cs { + doc.Containers = append(doc.Containers, c.Name) + } + ns, err := a.opts.Docker.Networks(ctx, byProject) + if err != nil { + return doc, err + } + for _, n := range ns { + doc.Networks = append(doc.Networks, n.Name) + } + if f.volumes { + vs, err := a.opts.Docker.Volumes(ctx, byProject) + if err != nil { + return doc, err + } + for _, v := range vs { + doc.Volumes = append(doc.Volumes, v.Name) + } + } + sort.Strings(doc.Containers) + sort.Strings(doc.Networks) + sort.Strings(doc.Volumes) + return doc, nil +} + +func printDownText(cmd *cobra.Command, doc downDocument, volumes bool) error { + var b strings.Builder + verb := "Removed" + if doc.DryRun { + verb = "Would remove" + } + fmt.Fprintf(&b, "%s from %s:\n", verb, doc.Stack.Project) + for _, list := range []struct { + kind string + names []string + }{{"container", doc.Containers}, {"network", doc.Networks}, {"volume", doc.Volumes}} { + for _, n := range list.names { + fmt.Fprintf(&b, " %s %s\n", list.kind, n) + } + } + if !volumes { + b.WriteString("Volumes stay; siding down --volumes removes them.\n") + } + if doc.DryRun { + b.WriteString("Dry run: nothing was removed.\n") + } + _, err := fmt.Fprint(cmd.OutOrStdout(), b.String()) + return err +} diff --git a/internal/cli/down_test.go b/internal/cli/down_test.go new file mode 100644 index 0000000..cbeee1e --- /dev/null +++ b/internal/cli/down_test.go @@ -0,0 +1,118 @@ +package cli + +import ( + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" + "github.com/tweinmann/siding/internal/testutil" +) + +// downEnv is a stack on branch with a network and a volume. +func downEnv(t *testing.T, branch string) *env { + t.Helper() + e := newEnv(t) + if branch != "main" { + e.git("switch", "-q", "-c", branch) + e.git("commit", "-q", "--allow-empty", "-m", "work") + } + id := e.id() + id.Branch = branch + e.proxyUp(80) + e.stackUp(id, "db", "web") + project := map[string]string{stack.LabelProject: id.Project()} + e.docker.networks = []docker.Network{{Name: id.Project() + "_default", Labels: project}, {Name: "siding", Labels: map[string]string{}}} + e.docker.volumes = []docker.Volume{{Name: id.Project() + "_data", Labels: project}, {Name: "shared-cache", Labels: map[string]string{}}} + return e +} + +func TestDown(t *testing.T) { + t.Parallel() + + t.Run("keeps the volumes", func(t *testing.T) { + t.Parallel() + e := downEnv(t, "main") + code, stdout, _ := e.run("down", "--json") + if code != exitOK { + t.Fatalf("exit %d: %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "down.json"), []byte(stdout)) + runs := e.compose.recorded() + if len(runs) != 1 || runs[0].inv.Project != "shop-main" || len(runs[0].inv.Files) != 0 || + !slices.Equal(runs[0].args, []string{"down", "--remove-orphans"}) { + t.Errorf("compose runs %+v", runs) + } + }) + + t.Run("dry run", func(t *testing.T) { + t.Parallel() + e := downEnv(t, "main") + code, stdout, _ := e.run("down", "--volumes", "--dry-run", "--json") + if code != exitOK || len(e.compose.recorded()) != 0 { + t.Fatalf("exit %d, compose runs %d", code, len(e.compose.recorded())) + } + testutil.Golden(t, filepath.Join("testdata", "json", "down-dryrun.json"), []byte(stdout)) + + _, text, _ := e.run("down", "--dry-run") + want := "Would remove from shop-main:\n container shop-main-db-1\n container shop-main-web-1\n network shop-main_default\n" + + "Volumes stay; siding down --volumes removes them.\nDry run: nothing was removed.\n" + if text != want { + t.Errorf("text:\n%s\nwant:\n%s", text, want) + } + }) + + t.Run("volumes of a merged, clean branch", func(t *testing.T) { + t.Parallel() + e := downEnv(t, "main") + if code, stdout, _ := e.run("down", "--volumes", "--json"); code != exitOK || !strings.Contains(stdout, "shop-main_data") { + t.Fatalf("exit %d: %s", code, stdout) + } + if runs := e.compose.recorded(); len(runs) != 1 || !slices.Equal(runs[0].args, []string{"down", "--remove-orphans", "--volumes"}) { + t.Errorf("compose runs %+v", runs) + } + }) + + for _, tt := range []struct { + name string + branch string + dirty bool + code string + }{ + {name: "unmerged branch", branch: "wip", code: fail.UnmergedBranch}, + {name: "dirty worktree", branch: "main", dirty: true, code: fail.DirtyWorktree}, + } { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + e := downEnv(t, tt.branch) + if tt.dirty { + e.write("notes.txt", "unsaved") + } + code, _, stderr := e.run("down", "--volumes") + if code != exitFailed || !strings.Contains(stderr, "--force") || len(e.compose.recorded()) != 0 { + t.Fatalf("exit %d, stderr %q, compose runs %d", code, stderr, len(e.compose.recorded())) + } + if _, stdout, _ := e.run("down", "--volumes", "--json"); !strings.Contains(stdout, `"code": "`+tt.code+`"`) { + t.Errorf("stdout %s, want code %s", stdout, tt.code) + } + // Without --volumes nothing is lost, so there is no guard. + if code, _, _ := e.run("down"); code != exitOK { + t.Errorf("down without --volumes: exit %d", code) + } + if code, _, _ := e.run("down", "--volumes", "--force"); code != exitOK { + t.Errorf("forced: exit %d", code) + } + }) + } + + t.Run("no stack", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + if code, stdout, _ := e.run("down", "--json"); code != exitFailed || !strings.Contains(stdout, `"code": "StackNotFound"`) { + t.Errorf("exit %d, stdout %s", code, stdout) + } + }) +} diff --git a/internal/cli/env_test.go b/internal/cli/env_test.go new file mode 100644 index 0000000..3cd23fb --- /dev/null +++ b/internal/cli/env_test.go @@ -0,0 +1,373 @@ +package cli + +import ( + "bytes" + "context" + "io" + "maps" + "os" + "os/exec" + "path/filepath" + "strings" + "sync" + "testing" + "time" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/probe" + "github.com/tweinmann/siding/internal/proxy" + "github.com/tweinmann/siding/internal/stack" +) + +// testConfig is what docker compose config would print for testCompose: web on / (one port), +// api on /api (a port label picks 8080 of two), db with no port, and mail with one exposed +// port but no route label. +const testConfig = `{ + "name": "shop-main", + "services": { + "api": {"expose": ["8080", "9229"], "labels": {"dev.siding.route": "/api", "dev.siding.port": "8080"}, + "networks": {"default": null}, "environment": {"API_TOKEN": "not-a-real-api-token"}}, + "db": {"networks": {"default": null}, "environment": {"POSTGRES_PASSWORD": "not-a-real-token"}}, + "mail": {"expose": ["8025"], "networks": {"default": null}}, + "web": {"ports": [{"target": 3000, "published": "3000", "protocol": "tcp"}], "labels": {"dev.siding.route": "/"}, + "networks": {"default": null}} + }, + "networks": {"default": {"name": "shop-main_default"}}, + "volumes": {"data": {"name": "shop-main_data"}} +}` + +// env is one test's machine: a real git repository /shop on main with a compose file, +// and fakes for everything else. +type env struct { + t *testing.T + base string // parent of the repository, shown as /src in golden files + repo string + opts Options + docker *fakeDocker + compose *fakeCompose + probe *fakeProbe + browser *fakeBrowser + clock *fakeClock + envVars map[string]string +} + +func newEnv(t *testing.T) *env { + t.Helper() + // The real path: git reports it, and on macOS the temporary directory is a symlink. + base, err := filepath.EvalSymlinks(t.TempDir()) + if err != nil { + t.Fatal(err) + } + e := &env{ + t: t, + base: base, + repo: filepath.Join(base, "shop"), + docker: &fakeDocker{logs: map[string][]string{}, memory: map[string]uint64{}}, + compose: &fakeCompose{config: testConfig}, + probe: &fakeProbe{answers: map[string]probe.Response{}}, + browser: &fakeBrowser{}, + clock: &fakeClock{now: time.Date(2026, 10, 28, 12, 0, 0, 0, time.UTC)}, + envVars: map[string]string{}, + } + if err := os.MkdirAll(e.repo, 0o755); err != nil { + t.Fatal(err) + } + e.git("init", "-q", "-b", "main") + e.write("compose.yaml", "services: {}\n") + e.write(".env", "POSTGRES_PASSWORD=not-a-real-token\n") + e.git("add", ".") + e.git("commit", "-q", "-m", "first") + e.opts = Options{ + Build: testBuild, + Dir: e.repo, + Home: filepath.Join(base, "home"), + Getenv: func(k string) string { return e.envVars[k] }, + OSMachineID: func() string { return "not-a-real-machine-id" }, + Git: git.Client{}, + Compose: e.compose, + Docker: e.docker, + Probe: e.probe, + Browser: e.browser, + Clock: e.clock, + Proxy: proxy.Default(), + } + return e +} + +func (e *env) git(args ...string) { + e.t.Helper() + cmd := exec.Command("git", append([]string{"-C", e.repo}, args...)...) + cmd.Env = append(os.Environ(), "GIT_CONFIG_GLOBAL=/dev/null", "GIT_CONFIG_NOSYSTEM=1", + "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@example.com", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@example.com") + if out, err := cmd.CombinedOutput(); err != nil { + e.t.Fatalf("git %v: %v\n%s", args, err, out) + } +} + +func (e *env) write(name, content string) { + e.t.Helper() + path := filepath.Join(e.repo, name) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + e.t.Fatal(err) + } + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + e.t.Fatal(err) + } +} + +// run runs siding with args and returns exit code, stdout and stderr, with the temporary +// directory shown as /src. +func (e *env) run(args ...string) (code int, stdout, stderr string) { + var out, errOut bytes.Buffer + code = Execute(context.Background(), New(e.opts), args, &out, &errOut) + return code, e.clean(out.String()), e.clean(errOut.String()) +} + +func (e *env) clean(s string) string { return strings.ReplaceAll(s, e.base, "/src") } + +// id is the identity siding gives the test repository. +func (e *env) id() stack.Identity { + machine, err := stack.MachineID("not-a-real-machine-id", "") + if err != nil { + e.t.Fatal(err) + } + return stack.Identity{Repo: "shop", Branch: "main", Worktree: e.repo, Machine: machine} +} + +// proxyUp makes siding-proxy run on port. +func (e *env) proxyUp(port int) { + e.docker.mu.Lock() + defer e.docker.mu.Unlock() + e.docker.proxy = &docker.Container{ID: "proxy", Name: "siding-proxy", Image: proxy.Image, State: "running", + Labels: map[string]string{}, Ports: []docker.Port{{HostIP: "127.0.0.1", HostPort: port, ContainerPort: 80}}} +} + +// stackUp adds one container per service of the test stack, running and, where routed, +// answering through the proxy. +func (e *env) stackUp(id stack.Identity, services ...string) { + for _, s := range services { + e.docker.add(container(id, s, "running", "", 0)) + } + e.probe.set("/", id.Project()+"/web", 200) + e.probe.set("/api", id.Project()+"/api", 404) +} + +func container(id stack.Identity, service, state, health string, exit int) docker.Container { + labels := id.ContainerLabels() + labels[stack.LabelProject] = id.Project() + labels[stack.LabelService] = service + return docker.Container{ID: id.Project() + "-" + service + "-1", Name: id.Project() + "-" + service + "-1", + Labels: labels, State: state, Health: health, ExitCode: exit} +} + +type fakeDocker struct { + mu sync.Mutex + pingErr error + proxy *docker.Container + containers []docker.Container + networks []docker.Network + volumes []docker.Volume + logs map[string][]string + memory map[string]uint64 + ensured []string +} + +func (d *fakeDocker) add(c docker.Container) { + d.mu.Lock() + defer d.mu.Unlock() + d.containers = append(d.containers, c) +} + +// update changes the container of a service. +func (d *fakeDocker) update(service string, f func(*docker.Container)) { + d.mu.Lock() + defer d.mu.Unlock() + for i := range d.containers { + if d.containers[i].Labels[stack.LabelService] == service { + f(&d.containers[i]) + } + } +} + +func matches(have, want map[string]string) bool { + for k, v := range want { + got, ok := have[k] + if !ok || (v != "" && got != v) { + return false + } + } + return true +} + +func (d *fakeDocker) Ping(context.Context) error { return d.pingErr } + +func (d *fakeDocker) Containers(_ context.Context, labels map[string]string) ([]docker.Container, error) { + d.mu.Lock() + defer d.mu.Unlock() + var out []docker.Container + for _, c := range d.containers { + if matches(c.Labels, labels) { + c.Labels = maps.Clone(c.Labels) + out = append(out, c) + } + } + return out, nil +} + +func (d *fakeDocker) Container(_ context.Context, name string) (*docker.Container, error) { + d.mu.Lock() + defer d.mu.Unlock() + if d.proxy != nil && d.proxy.Name == name { + c := *d.proxy + return &c, nil + } + return nil, nil +} + +func (d *fakeDocker) Networks(_ context.Context, labels map[string]string) ([]docker.Network, error) { + d.mu.Lock() + defer d.mu.Unlock() + var out []docker.Network + for _, n := range d.networks { + if matches(n.Labels, labels) { + out = append(out, n) + } + } + return out, nil +} + +func (d *fakeDocker) Volumes(_ context.Context, labels map[string]string) ([]docker.Volume, error) { + d.mu.Lock() + defer d.mu.Unlock() + var out []docker.Volume + for _, v := range d.volumes { + if matches(v.Labels, labels) { + out = append(out, v) + } + } + return out, nil +} + +func (d *fakeDocker) EnsureNetwork(_ context.Context, name string, _ map[string]string) (bool, error) { + d.mu.Lock() + defer d.mu.Unlock() + d.ensured = append(d.ensured, name) + return true, nil +} + +func (d *fakeDocker) Logs(_ context.Context, id string, tail int) ([]string, error) { + d.mu.Lock() + defer d.mu.Unlock() + lines := d.logs[id] + if len(lines) > tail { + lines = lines[len(lines)-tail:] + } + return lines, nil +} + +func (d *fakeDocker) Memory(_ context.Context, id string) (uint64, error) { + d.mu.Lock() + defer d.mu.Unlock() + return d.memory[id], nil +} + +type composeRun struct { + inv compose.Invocation + args []string +} + +type fakeCompose struct { + mu sync.Mutex + config string + runs []composeRun + runErr error + // onRun, if set, runs after each recorded run, e.g. to start containers. + onRun func(composeRun) +} + +func (c *fakeCompose) Config(context.Context, compose.Invocation) ([]byte, error) { + return []byte(c.config), nil +} + +func (c *fakeCompose) Run(_ context.Context, inv compose.Invocation, args []string, _, _ io.Writer) error { + c.mu.Lock() + r := composeRun{inv: inv, args: args} + c.runs = append(c.runs, r) + onRun, err := c.onRun, c.runErr + c.mu.Unlock() + if err != nil { + return err + } + if onRun != nil { + onRun(r) + } + return nil +} + +func (c *fakeCompose) recorded() []composeRun { + c.mu.Lock() + defer c.mu.Unlock() + return append([]composeRun(nil), c.runs...) +} + +type fakeProbe struct { + mu sync.Mutex + answers map[string]probe.Response // by path + asked []string +} + +func (p *fakeProbe) set(path, service string, status int) { + p.mu.Lock() + defer p.mu.Unlock() + p.answers[path] = probe.Response{Status: status, Service: service} +} + +func (p *fakeProbe) Get(_ context.Context, addr, host, path string) (probe.Response, error) { + p.mu.Lock() + defer p.mu.Unlock() + p.asked = append(p.asked, addr+" "+host+path) + if r, ok := p.answers[path]; ok { + return r, nil + } + return probe.Response{Status: 404}, nil // like Traefik without a router +} + +type fakeBrowser struct { + mu sync.Mutex + opened []string +} + +func (b *fakeBrowser) Open(url string) error { + b.mu.Lock() + defer b.mu.Unlock() + b.opened = append(b.opened, url) + return nil +} + +// fakeClock advances only when slept on, and runs onSleep after each sleep. +type fakeClock struct { + mu sync.Mutex + now time.Time + slept int + onSleep func(slept int) +} + +func (c *fakeClock) Now() time.Time { + c.mu.Lock() + defer c.mu.Unlock() + return c.now +} + +func (c *fakeClock) Sleep(_ context.Context, d time.Duration) error { + c.mu.Lock() + c.now = c.now.Add(d) + c.slept++ + n, f := c.slept, c.onSleep + c.mu.Unlock() + if f != nil { + f(n) + } + return nil +} diff --git a/internal/cli/json.go b/internal/cli/json.go index 02b8455..be95652 100644 --- a/internal/cli/json.go +++ b/internal/cli/json.go @@ -18,8 +18,16 @@ import ( const documentAPIVersion = "siding.dev/v1alpha1" const ( - kindError = "Error" - kindVersion = "Version" + kindAddress = "Address" + kindDown = "Down" + kindError = "Error" + kindLogs = "Logs" + kindSetup = "Setup" + kindSkill = "Skill" + kindStack = "Stack" + kindStackList = "StackList" + kindStatus = "Status" + kindVersion = "Version" ) // jsonFlag is the name of the flag, looked up again by Execute to report an error as a document. @@ -58,6 +66,7 @@ func wantsJSON(cmd *cobra.Command, args []string) bool { func printJSON(w io.Writer, doc any) error { enc := json.NewEncoder(w) enc.SetIndent("", " ") + enc.SetEscapeHTML(false) return enc.Encode(doc) } @@ -66,6 +75,7 @@ type errorDocument struct { header Code string `json:"code"` Message string `json:"message"` + Hint string `json:"hint,omitempty"` } type versionDocument struct { @@ -75,3 +85,127 @@ type versionDocument struct { GoVersion string `json:"goVersion"` Platform string `json:"platform"` } + +// stackRef names a stack in other documents. +type stackRef struct { + Project string `json:"project"` + Repo string `json:"repo"` + Branch string `json:"branch"` + Worktree string `json:"worktree"` + // URL is the stack's address: its host on the proxy's port. + URL string `json:"url,omitempty"` +} + +// setupDocument is what setup prints. +type setupDocument struct { + header + Container string `json:"container"` + Network string `json:"network"` + Image string `json:"image"` + Port int `json:"port"` + URL string `json:"url"` + // State is created, updated or unchanged. + State string `json:"state"` +} + +// stackDocument is what up prints without --wait. +type stackDocument struct { + header + Stack stackRef `json:"stack"` + Services []serviceAddress `json:"services,omitempty"` +} + +type serviceAddress struct { + Name string `json:"name"` + // URL is the service host. + URL string `json:"url"` + // Route is the path on the stack's host, if any. + Route string `json:"route,omitempty"` + Port int `json:"port"` +} + +// statusDocument is the verdict of status and up --wait. +type statusDocument struct { + header + // Verdict is Ready, Starting, Failed or TimedOut. + Verdict string `json:"verdict"` + Stack stackRef `json:"stack"` + Services []serviceStatus `json:"services,omitempty"` + Failure *failureDocument `json:"failure,omitempty"` +} + +type serviceStatus struct { + Name string `json:"name"` + State string `json:"state"` + Health string `json:"health,omitempty"` + Route string `json:"route,omitempty"` + URL string `json:"url,omitempty"` + // Answering is set for services with a route path. + Answering *bool `json:"answering,omitempty"` + Ready bool `json:"ready"` +} + +type failureDocument struct { + Service string `json:"service"` + // Reason is Exited, Unhealthy, Restarting, Missing, NotAnswering or NotReady. + Reason string `json:"reason"` + Hint string `json:"hint"` + Logs []string `json:"logs,omitempty"` +} + +// stackListDocument is what ps prints. +type stackListDocument struct { + header + Stacks []stackEntry `json:"stacks,omitempty"` + MemoryBytes uint64 `json:"memoryBytes,omitempty"` +} + +type stackEntry struct { + Project string `json:"project"` + Repo string `json:"repo"` + Branch string `json:"branch"` + Worktree string `json:"worktree"` + Machine string `json:"machine,omitempty"` + Agent string `json:"agent,omitempty"` + // State is running, partial or stopped. + State string `json:"state"` + URL string `json:"url"` + MemoryBytes uint64 `json:"memoryBytes,omitempty"` + WorktreeMissing bool `json:"worktreeMissing,omitempty"` +} + +// logsDocument is what logs --json prints. +type logsDocument struct { + header + Stack stackRef `json:"stack"` + Services []serviceLogs `json:"services,omitempty"` +} + +type serviceLogs struct { + Name string `json:"name"` + Lines []string `json:"lines,omitempty"` +} + +// addressDocument is what open --json prints. +type addressDocument struct { + header + URL string `json:"url"` + Service string `json:"service,omitempty"` +} + +// downDocument is what down prints. +type downDocument struct { + header + Stack stackRef `json:"stack"` + DryRun bool `json:"dryRun,omitempty"` + Containers []string `json:"containers,omitempty"` + Networks []string `json:"networks,omitempty"` + Volumes []string `json:"volumes,omitempty"` +} + +// skillDocument is what skill --json prints. +type skillDocument struct { + header + Name string `json:"name"` + Text string `json:"text"` +} diff --git a/internal/cli/logs.go b/internal/cli/logs.go new file mode 100644 index 0000000..5d531fa --- /dev/null +++ b/internal/cli/logs.go @@ -0,0 +1,94 @@ +package cli + +import ( + "slices" + "strconv" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" +) + +// defaultJSONTail is how many lines per service logs --json prints without --tail. +const defaultJSONTail = 100 + +func newLogsCommand(a *app) *cobra.Command { + var asJSON, follow bool + var tail int + var file string + cmd := &cobra.Command{ + Use: "logs [service...]", + Short: "Show the logs of the worktree's stack", + Long: `Show the logs of the worktree's stack, of all services or of the ones named. With +--json it prints the last lines per service (100 without --tail) as one document, with the +values of .env and of the services' environment replaced by ***.`, + RunE: func(cmd *cobra.Command, services []string) error { + if asJSON && follow { + return fail.New(fail.InvalidArguments, "--json and --follow cannot be combined: --json prints one document") + } + ctx := cmd.Context() + st, err := a.findStack(ctx) + if err != nil { + return err + } + if !asJSON { + args := []string{"logs"} + if follow { + args = append(args, "--follow") + } + if tail > 0 { + args = append(args, "--tail", strconv.Itoa(tail)) + } + inv := compose.Invocation{Dir: st.wt.Top, Project: st.project} + return a.opts.Compose.Run(ctx, inv, append(args, services...), cmd.OutOrStdout(), cmd.ErrOrStderr()) + } + + if tail <= 0 { + tail = defaultJSONTail + } + _, p, err := a.loadProject(ctx, st.wt.Top, file, st.project) + if err != nil { + return err + } + r, err := redactor(st.wt.Top, p) + if err != nil { + return err + } + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelProject: st.project}) + if err != nil { + return err + } + grouped := byService(cs) + names := services + if len(names) == 0 { + names = p.ServiceNames() + } + doc := logsDocument{header: newHeader(kindLogs), Stack: st.ref(a.shownPort(ctx))} + for _, name := range names { + if _, ok := p.Services[name]; !ok { + return fail.New(fail.InvalidArguments, "the stack has no service %q", name). + WithHint("its services are %v", p.ServiceNames()) + } + sl := serviceLogs{Name: name} + for _, c := range grouped[name] { + lines, err := a.opts.Docker.Logs(ctx, c.ID, tail) + if err != nil { + return err + } + sl.Lines = append(sl.Lines, r.Lines(lines)...) + } + doc.Services = append(doc.Services, sl) + } + slices.SortFunc(doc.Services, func(x, y serviceLogs) int { return strings.Compare(x.Name, y.Name) }) + return printJSON(cmd.OutOrStdout(), doc) + }, + } + cmd.Flags().BoolVarP(&follow, "follow", "f", false, "keep printing new lines") + cmd.Flags().IntVar(&tail, "tail", 0, "only the last n lines of each service") + cmd.Flags().StringVar(&file, "file", "", "compose file (default: compose.yaml or another name Compose looks for)") + registerJSON(cmd, &asJSON) + return cmd +} diff --git a/internal/cli/logs_test.go b/internal/cli/logs_test.go new file mode 100644 index 0000000..c17b652 --- /dev/null +++ b/internal/cli/logs_test.go @@ -0,0 +1,62 @@ +package cli + +import ( + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/testutil" +) + +func TestLogs(t *testing.T) { + t.Parallel() + + t.Run("json, redacted", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "db", "mail", "web") + e.docker.logs["shop-main-api-1"] = []string{"one", "token not-a-real-api-token accepted", "three"} + e.docker.logs["shop-main-db-1"] = []string{"listening on 5432", "password not-a-real-token"} + code, stdout, _ := e.run("logs", "--json", "--tail", "2", "api", "db") + if code != exitOK { + t.Fatalf("exit %d: %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "logs.json"), []byte(stdout)) + if strings.Contains(stdout, "not-a-real") { + t.Error("a secret reached the document") + } + }) + + t.Run("text goes through compose", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.stackUp(e.id(), "api") + if code, _, stderr := e.run("logs", "-f", "--tail", "5", "api"); code != exitOK { + t.Fatalf("exit %d: %s", code, stderr) + } + runs := e.compose.recorded() + if len(runs) != 1 || runs[0].inv.Project != "shop-main" || len(runs[0].inv.Files) != 0 || + !slices.Equal(runs[0].args, []string{"logs", "--follow", "--tail", "5", "api"}) { + t.Errorf("compose runs %+v", runs) + } + }) + + t.Run("json and follow", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + if code, stdout, _ := e.run("logs", "--json", "-f"); code != exitFailed || !strings.Contains(stdout, `"code": "InvalidArguments"`) { + t.Errorf("exit %d, stdout %s", code, stdout) + } + }) + + t.Run("unknown service", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.stackUp(e.id(), "api") + if code, stdout, _ := e.run("logs", "--json", "nope"); code != exitFailed || !strings.Contains(stdout, `"code": "InvalidArguments"`) { + t.Errorf("exit %d, stdout %s", code, stdout) + } + }) +} diff --git a/internal/cli/open.go b/internal/cli/open.go new file mode 100644 index 0000000..d778b5a --- /dev/null +++ b/internal/cli/open.go @@ -0,0 +1,85 @@ +package cli + +import ( + "fmt" + "os/exec" + "runtime" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" +) + +func newOpenCommand(a *app) *cobra.Command { + var asJSON, printOnly bool + var file string + cmd := &cobra.Command{ + Use: "open [service]", + Short: "Open the stack's address, or a service's, in the browser", + Args: cobra.MaximumNArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + ctx := cmd.Context() + st, err := a.findStack(ctx) + if err != nil { + return err + } + port := a.shownPort(ctx) + doc := addressDocument{header: newHeader(kindAddress), URL: address(st.id.Host(), port, "/")} + if len(args) == 1 { + _, p, err := a.loadProject(ctx, st.wt.Top, file, st.project) + if err != nil { + return err + } + routes, _, err := stack.Routes(p) + if err != nil { + return err + } + if _, ok := stack.Find(routes, args[0]); !ok { + names := make([]string, len(routes)) + for i, r := range routes { + names[i] = r.Service + } + return fail.New(fail.InvalidArguments, "the stack has no HTTP service %q", args[0]). + WithHint("its HTTP services are: %s", strings.Join(names, ", ")) + } + doc.Service = args[0] + doc.URL = address(st.id.ServiceHost(args[0]), port, "/") + } + switch { + case asJSON: + return printJSON(cmd.OutOrStdout(), doc) + case printOnly: + _, err := fmt.Fprintln(cmd.OutOrStdout(), doc.URL) + return err + } + if err := a.opts.Browser.Open(doc.URL); err != nil { + return fail.New(fail.Internal, "open %s: %v", doc.URL, err).WithHint("open it yourself, or print it with --print") + } + _, err = fmt.Fprintln(cmd.OutOrStdout(), doc.URL) + return err + }, + } + cmd.Flags().BoolVar(&printOnly, "print", false, "print the address instead of opening it") + cmd.Flags().StringVarP(&file, "file", "f", "", "compose file (default: compose.yaml or another name Compose looks for)") + registerJSON(cmd, &asJSON) + return cmd +} + +// SystemBrowser opens URLs with the platform's opener: open on macOS, $BROWSER (set by VS Code +// in a devcontainer) or xdg-open on Linux. +type SystemBrowser struct { + Getenv func(string) string +} + +// Open opens url. +func (b SystemBrowser) Open(url string) error { + if runtime.GOOS == "darwin" { + return exec.Command("open", url).Run() + } + if browser := b.Getenv("BROWSER"); browser != "" { + return exec.Command(browser, url).Run() + } + return exec.Command("xdg-open", url).Run() +} diff --git a/internal/cli/open_test.go b/internal/cli/open_test.go new file mode 100644 index 0000000..be4f890 --- /dev/null +++ b/internal/cli/open_test.go @@ -0,0 +1,68 @@ +package cli + +import ( + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/testutil" +) + +func TestOpen(t *testing.T) { + t.Parallel() + + for _, tt := range []struct { + name string + port int + args []string + want string + }{ + {name: "port 80", port: 80, args: []string{"open", "--print"}, want: "http://shop.main.localhost/\n"}, + {name: "port 8080", port: 8080, args: []string{"open", "--print"}, want: "http://shop.main.localhost:8080/\n"}, + {name: "a service", port: 8080, args: []string{"open", "--print", "mail"}, want: "http://mail.shop.main.localhost:8080/\n"}, + } { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(tt.port) + e.stackUp(e.id(), "web") + if code, stdout, _ := e.run(tt.args...); code != exitOK || stdout != tt.want || len(e.browser.opened) != 0 { + t.Errorf("exit %d, stdout %q, opened %v; want %q and no browser", code, stdout, e.browser.opened, tt.want) + } + }) + } + + t.Run("opens the browser", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "web") + if code, _, _ := e.run("open"); code != exitOK || !slices.Equal(e.browser.opened, []string{"http://shop.main.localhost/"}) { + t.Errorf("exit %d, opened %v", code, e.browser.opened) + } + }) + + t.Run("json", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "web") + code, stdout, _ := e.run("open", "api", "--json") + if code != exitOK || len(e.browser.opened) != 0 { + t.Fatalf("exit %d, opened %v", code, e.browser.opened) + } + testutil.Golden(t, filepath.Join("testdata", "json", "address.json"), []byte(stdout)) + }) + + t.Run("unknown service", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "web") + code, _, stderr := e.run("open", "nope") + if code != exitFailed || !strings.Contains(stderr, `no HTTP service "nope"`) || !strings.Contains(stderr, "api, mail, web") { + t.Errorf("exit %d, stderr %q", code, stderr) + } + }) +} diff --git a/internal/cli/ps.go b/internal/cli/ps.go new file mode 100644 index 0000000..47dc6b8 --- /dev/null +++ b/internal/cli/ps.go @@ -0,0 +1,144 @@ +package cli + +import ( + "context" + "errors" + "fmt" + "io/fs" + "os" + "sort" + "sync" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/stack" +) + +func newPsCommand(a *app) *cobra.Command { + var asJSON bool + cmd := &cobra.Command{ + Use: "ps", + Short: "List every stack on this machine with its state, address and memory", + Long: `List every stack on this machine, from any directory: repository, branch, state +(running, partial or stopped), address, memory in use and worktree. ps reads only Docker; the +full verdict on one stack is siding status.`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + doc, err := a.ps(cmd.Context()) + if err != nil { + return err + } + if asJSON { + return printJSON(cmd.OutOrStdout(), doc) + } + return printPsText(cmd, doc) + }, + } + registerJSON(cmd, &asJSON) + return cmd +} + +func (a *app) ps(ctx context.Context) (stackListDocument, error) { + doc := stackListDocument{header: newHeader(kindStackList)} + if err := a.opts.Docker.Ping(ctx); err != nil { + return doc, err + } + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelRepo: ""}) + if err != nil { + return doc, err + } + port := a.shownPort(ctx) + projects := map[string][]docker.Container{} + for _, c := range cs { + p := c.Labels[stack.LabelProject] + projects[p] = append(projects[p], c) + } + memory, err := a.memory(ctx, cs) + if err != nil { + return doc, err + } + for project, containers := range projects { + id := stack.FromLabels(containers[0].Labels) + e := stackEntry{Project: project, Repo: id.Repo, Branch: id.Branch, Worktree: id.Worktree, + Machine: id.Machine, Agent: id.Agent, URL: address(id.Host(), port, "/")} + running := 0 + for _, c := range containers { + if c.State == "running" { + running++ + } + e.MemoryBytes += memory[c.ID] + } + switch running { + case len(containers): + e.State = "running" + case 0: + e.State = "stopped" + default: + e.State = "partial" + } + if _, err := os.Stat(id.Worktree); errors.Is(err, fs.ErrNotExist) { + e.WorktreeMissing = true + } + doc.MemoryBytes += e.MemoryBytes + doc.Stacks = append(doc.Stacks, e) + } + sort.Slice(doc.Stacks, func(i, j int) bool { return doc.Stacks[i].Project < doc.Stacks[j].Project }) + return doc, nil +} + +// memory asks for the memory of every running container at once; a stats call takes a moment. +func (a *app) memory(ctx context.Context, cs []docker.Container) (map[string]uint64, error) { + var mu sync.Mutex + var wg sync.WaitGroup + out := map[string]uint64{} + var firstErr error + for _, c := range cs { + if c.State != "running" { + continue + } + wg.Add(1) + go func(id string) { + defer wg.Done() + m, err := a.opts.Docker.Memory(ctx, id) + mu.Lock() + defer mu.Unlock() + if err != nil { + if firstErr == nil { + firstErr = err + } + return + } + out[id] = m + }(c.ID) + } + wg.Wait() + return out, firstErr +} + +func printPsText(cmd *cobra.Command, doc stackListDocument) error { + if len(doc.Stacks) == 0 { + _, err := fmt.Fprintln(cmd.OutOrStdout(), "No stacks. Start one with siding up in a worktree.") + return err + } + rows := [][]string{{"REPO", "BRANCH", "STATE", "URL", "MEMORY", "WORKTREE"}} + for _, s := range doc.Stacks { + worktree := s.Worktree + if s.WorktreeMissing { + worktree += " (missing)" + } + rows = append(rows, []string{s.Repo, s.Branch, s.State, s.URL, bytesText(s.MemoryBytes), worktree}) + } + rows = append(rows, []string{"total", "", "", "", bytesText(doc.MemoryBytes), ""}) + _, err := fmt.Fprint(cmd.OutOrStdout(), table("", rows)) + return err +} + +// bytesText shows memory in MiB, or GiB from 1 GiB on. +func bytesText(n uint64) string { + const mib, gib = 1 << 20, 1 << 30 + if n >= gib { + return fmt.Sprintf("%.1f GiB", float64(n)/gib) + } + return fmt.Sprintf("%d MiB", (n+mib/2)/mib) +} diff --git a/internal/cli/ps_test.go b/internal/cli/ps_test.go new file mode 100644 index 0000000..aa2e10f --- /dev/null +++ b/internal/cli/ps_test.go @@ -0,0 +1,70 @@ +package cli + +import ( + "path/filepath" + "testing" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/testutil" +) + +func TestPs(t *testing.T) { + t.Parallel() + + t.Run("stacks", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(8080) + main := e.id() + login := e.id() + login.Branch, login.Worktree, login.Agent = "feature/login", filepath.Join(e.base, "shop-login"), "claude" + e.stackUp(main, "web", "db") + e.stackUp(login, "web", "db") + e.docker.update("db", func(c *docker.Container) { + if c.Labels["dev.siding.branch"] == "feature/login" { + c.State = "exited" + } + }) + e.docker.add(docker.Container{ID: "foreign", Labels: map[string]string{"com.docker.compose.project": "other"}, State: "running"}) + const mib = 1 << 20 + e.docker.memory["shop-main-web-1"] = 200 * mib + e.docker.memory["shop-main-db-1"] = 100 * mib + e.docker.memory["shop-feature-login-web-1"] = 500 * mib + + code, stdout, _ := e.run("ps", "--json") + if code != exitOK { + t.Fatalf("exit %d", code) + } + testutil.Golden(t, filepath.Join("testdata", "json", "stacklist.json"), []byte(stdout)) + + _, text, _ := e.run("ps") + want := `REPO BRANCH STATE URL MEMORY WORKTREE +shop feature/login partial http://shop.feature-login.localhost:8080/ 500 MiB /src/shop-login (missing) +shop main running http://shop.main.localhost:8080/ 300 MiB /src/shop +total 800 MiB +` + if text != want { + t.Errorf("text:\n%s\nwant:\n%s", text, want) + } + }) + + t.Run("no stacks", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + code, stdout, _ := e.run("ps", "--json") + if code != exitOK { + t.Fatalf("exit %d", code) + } + testutil.Golden(t, filepath.Join("testdata", "json", "stacklist-empty.json"), []byte(stdout)) + }) + + t.Run("from outside any repository", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.stackUp(e.id(), "web") + e.opts.Dir = t.TempDir() + if code, stdout, _ := e.run("ps"); code != exitOK || stdout == "" { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) +} diff --git a/internal/cli/setup.go b/internal/cli/setup.go new file mode 100644 index 0000000..6fc003e --- /dev/null +++ b/internal/cli/setup.go @@ -0,0 +1,120 @@ +package cli + +import ( + "errors" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/proxy" + "github.com/tweinmann/siding/internal/stack" +) + +func newSetupCommand(a *app) *cobra.Command { + var asJSON bool + var port int + cmd := &cobra.Command{ + Use: "setup", + Short: "Start the shared proxy siding-proxy on 127.0.0.1", + Long: `Start the shared proxy siding-proxy: Traefik on the network "siding", published on +127.0.0.1 only. Without --port it keeps the port of a running proxy, and uses 80 otherwise. +Running setup again changes nothing unless the port or the image differs.`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + if port < 0 || port > 65535 { + return fail.New(fail.InvalidArguments, "--port must be between 1 and 65535, got %d", port) + } + doc, err := a.setup(cmd, port) + if err != nil { + return err + } + if asJSON { + return printJSON(cmd.OutOrStdout(), doc) + } + verb := map[string]string{"created": "started", "updated": "restarted", "unchanged": "already runs"}[doc.State] + _, err = fmt.Fprintf(cmd.OutOrStdout(), "%s %s on %s\n", doc.Container, verb, address("*.localhost", doc.Port, "/")) + return err + }, + } + cmd.Flags().IntVar(&port, "port", 0, "port on 127.0.0.1 for the proxy (default: the running proxy's, else 80)") + registerJSON(cmd, &asJSON) + return cmd +} + +func (a *app) setup(cmd *cobra.Command, port int) (setupDocument, error) { + ctx := cmd.Context() + cfg := a.opts.Proxy + if err := a.opts.Docker.Ping(ctx); err != nil { + return setupDocument{}, err + } + machine, err := a.machineID() + if err != nil { + return setupDocument{}, err + } + current, err := a.opts.Docker.Container(ctx, cfg.Container) + if err != nil { + return setupDocument{}, err + } + currentPort := 0 + if current != nil { + for _, p := range current.Ports { + if p.ContainerPort == proxyContainerPort { + currentPort = p.HostPort + } + } + } + if port == 0 { + port = cfg.DefaultPort + if current != nil && current.State == "running" && currentPort != 0 { + port = currentPort + } + } + doc := setupDocument{header: newHeader(kindSetup), Container: cfg.Container, Network: cfg.Network, + Image: cfg.Image, Port: port, URL: address("*.localhost", port, "/")} + + if current != nil && current.State == "running" && currentPort == port && current.Image == cfg.Image { + doc.State = "unchanged" + return doc, nil + } + doc.State = "created" + if current != nil { + doc.State = "updated" + } + + if _, err := a.opts.Docker.EnsureNetwork(ctx, cfg.Network, map[string]string{stack.LabelMachine: machine}); err != nil { + return setupDocument{}, err + } + dir := filepath.Join(a.opts.Home, ".siding", "proxy", cfg.Project) + if err := os.MkdirAll(dir, 0o755); err != nil { + return setupDocument{}, fmt.Errorf("write the proxy's compose file: %w", err) + } + file := filepath.Join(dir, "compose.yml") + if err := os.WriteFile(file, proxy.ComposeFile(cfg, port, machine), 0o644); err != nil { + return setupDocument{}, fmt.Errorf("write the proxy's compose file: %w", err) + } + inv := compose.Invocation{Dir: dir, Project: cfg.Project, Files: []string{file}} + err = a.opts.Compose.Run(ctx, inv, []string{"up", "-d", "--wait", "--wait-timeout", "60", "--remove-orphans"}, cmd.ErrOrStderr(), cmd.ErrOrStderr()) + if err != nil { + return setupDocument{}, portInUse(err, port) + } + return doc, nil +} + +// portInUse turns Compose's "address already in use" into PortInUse. +func portInUse(err error, port int) error { + var e *fail.Error + if errors.As(err, &e) { + msg := strings.ToLower(e.Message) + if strings.Contains(msg, "address already in use") || strings.Contains(msg, "port is already allocated") || + strings.Contains(msg, "ports are not available") { + return fail.New(fail.PortInUse, "port %d on 127.0.0.1 is in use", port). + WithHint("free it, or choose another port with siding setup --port 8080") + } + } + return err +} diff --git a/internal/cli/setup_test.go b/internal/cli/setup_test.go new file mode 100644 index 0000000..0d7a1f3 --- /dev/null +++ b/internal/cli/setup_test.go @@ -0,0 +1,85 @@ +package cli + +import ( + "os" + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/testutil" +) + +func TestSetup(t *testing.T) { + t.Parallel() + + t.Run("first setup", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + code, stdout, stderr := e.run("setup", "--json") + if code != exitOK || stderr != "" { + t.Fatalf("exit %d, stderr %q", code, stderr) + } + testutil.Golden(t, filepath.Join("testdata", "json", "setup.json"), []byte(stdout)) + if !slices.Equal(e.docker.ensured, []string{"siding"}) { + t.Errorf("networks ensured: %v", e.docker.ensured) + } + runs := e.compose.recorded() + if len(runs) != 1 || runs[0].inv.Project != "siding" || runs[0].args[0] != "up" { + t.Fatalf("compose runs %+v", runs) + } + data, err := os.ReadFile(runs[0].inv.Files[0]) + if err != nil { + t.Fatal(err) + } + for _, want := range []string{`"127.0.0.1:80:80"`, `container_name: "siding-proxy"`, "LabelRegex(`dev.siding.repo`, `.+`)", + "Label(`traefik.docker.network`, `siding`)", "--providers.docker.exposedbydefault=false"} { + if !strings.Contains(string(data), want) { + t.Errorf("the proxy's compose file lacks %s:\n%s", want, data) + } + } + if !strings.HasPrefix(runs[0].inv.Files[0], e.opts.Home+"/.siding/proxy/") { + t.Errorf("compose file %s is not under ~/.siding", runs[0].inv.Files[0]) + } + }) + + t.Run("already running", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + code, stdout, _ := e.run("setup") + if code != exitOK || stdout != "siding-proxy already runs on http://*.localhost/\n" || len(e.compose.recorded()) != 0 { + t.Errorf("exit %d, stdout %q, compose runs %d", code, stdout, len(e.compose.recorded())) + } + }) + + t.Run("keeps the running port without --port", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(8080) + if code, stdout, _ := e.run("setup"); code != exitOK || !strings.Contains(stdout, "already runs on http://*.localhost:8080/") { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) + + t.Run("another port recreates the proxy", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + code, stdout, _ := e.run("setup", "--port", "8080") + if code != exitOK || stdout != "siding-proxy restarted on http://*.localhost:8080/\n" || len(e.compose.recorded()) != 1 { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) + + t.Run("port in use", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.compose.runErr = fail.New(fail.ComposeFailed, "docker compose up failed: Error response from daemon: failed to bind host port 127.0.0.1:80/tcp: address already in use") + code, _, stderr := e.run("setup") + if code != exitFailed || !strings.Contains(stderr, "port 80 on 127.0.0.1 is in use") || !strings.Contains(stderr, "--port") { + t.Errorf("exit %d, stderr %q", code, stderr) + } + }) +} diff --git a/internal/cli/skill.go b/internal/cli/skill.go new file mode 100644 index 0000000..1b8b8cf --- /dev/null +++ b/internal/cli/skill.go @@ -0,0 +1,36 @@ +package cli + +import ( + _ "embed" + "io" + + "github.com/spf13/cobra" +) + +// skillText is the Claude Code skill siding ships, kept in the binary so that text and binary +// never drift apart. +// +//go:embed skill/SKILL.md +var skillText string + +func newSkillCommand() *cobra.Command { + var asJSON bool + cmd := &cobra.Command{ + Use: "skill", + Short: "Print a Claude Code skill that teaches an agent to use siding", + Long: `Print a Claude Code skill (SKILL.md) that tells an agent to start its stack and wait +for the verdict. siding writes no file; install it with + + mkdir -p .claude/skills/siding && siding skill > .claude/skills/siding/SKILL.md`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + if asJSON { + return printJSON(cmd.OutOrStdout(), skillDocument{header: newHeader(kindSkill), Name: "siding", Text: skillText}) + } + _, err := io.WriteString(cmd.OutOrStdout(), skillText) + return err + }, + } + registerJSON(cmd, &asJSON) + return cmd +} diff --git a/internal/cli/skill/SKILL.md b/internal/cli/skill/SKILL.md new file mode 100644 index 0000000..2f6d9c6 --- /dev/null +++ b/internal/cli/skill/SKILL.md @@ -0,0 +1,51 @@ +--- +name: siding +description: Start this worktree's Docker Compose stack with siding and wait for the verdict, to check that a change to the app actually runs. Use after changing code, Dockerfiles, compose files or configuration that the running app depends on, and before reporting such work as done. +--- + +# Check that the app runs with siding + +siding runs this git worktree's compose file as its own stack, reachable through a local proxy +at `http://..localhost`, and tells you whether it actually runs. Every command +below prints one JSON document with `--json`. + +## After a change that affects the running app + +Run: + +```sh +siding up --wait --json +``` + +It starts or updates the stack and waits, three minutes at most, for one `Status` document. +Read its `verdict` and the exit code: + +- **0, `Ready`**: every service runs, every healthcheck is healthy and every routed service + answers through the proxy. Report the address in `stack.url` (and a service's `url` if it + matters) together with your result. +- **1, `Failed`**: `failure` names the `service`, the `reason` (`Exited`, `Unhealthy`, + `Restarting`, `Missing`), a `hint` and the last log lines of that service in `logs`. Fix the + cause in that service, then run `siding up --wait --json` again. +- **2, `TimedOut`**: the stack was not ready in time. `failure` names the first service that + was not ready: `NotAnswering` means it runs but does not answer through the proxy (often it + listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady` + means it is still starting. Read the hint and the logs; if it is only slow, run + `siding status --wait --json --timeout 5m`. + +An `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`, +`message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`. + +## More detail + +- `siding status --json`: the current verdict, without waiting (exit 2 while starting). +- `siding logs --json` or `siding logs --json --tail 200`: the last log lines per + service. Values from `.env` and the services' environment appear as `***`. +- `siding ps --json`: every stack on this machine, with state and memory. + +## Rules + +- Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file + or the service labels `dev.siding.route` and `dev.siding.port` instead. +- Leave stopping and removing stacks to the user; `siding down` only removes containers, and + removing data is the user's decision. +- Do not paste log lines into places outside this conversation. diff --git a/internal/cli/skill_test.go b/internal/cli/skill_test.go new file mode 100644 index 0000000..e94cd55 --- /dev/null +++ b/internal/cli/skill_test.go @@ -0,0 +1,45 @@ +package cli + +import ( + "encoding/json" + "path/filepath" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/testutil" +) + +func TestSkill(t *testing.T) { + t.Parallel() + e := newEnv(t) + code, text, _ := e.run("skill") + if code != exitOK { + t.Fatalf("exit %d", code) + } + testutil.Golden(t, filepath.Join("testdata", "skill", "SKILL.md"), []byte(text)) + + if !strings.HasPrefix(text, "---\nname: siding\ndescription: ") { + t.Errorf("not a skill named siding:\n%s", text[:min(len(text), 80)]) + } + for _, want := range []string{"siding up --wait --json", "siding status --json", "siding logs --json", "**0, `Ready`**", "**1, `Failed`**", "**2, `TimedOut`**"} { + if !strings.Contains(text, want) { + t.Errorf("the skill does not mention %q", want) + } + } + // Only a human removes data, forces anything, or exposes a stack. + for _, never := range []string{"--force", "--volumes", "share"} { + if strings.Contains(text, never) { + t.Errorf("the skill mentions %q", never) + } + } + + code, stdout, _ := e.run("skill", "--json") + if code != exitOK { + t.Fatalf("exit %d", code) + } + testutil.Golden(t, filepath.Join("testdata", "json", "skill.json"), []byte(stdout)) + var doc skillDocument + if err := json.Unmarshal([]byte(stdout), &doc); err != nil || doc.Text != text || doc.Name != "siding" { + t.Errorf("document %+v, err %v", doc, err) + } +} diff --git a/internal/cli/stack.go b/internal/cli/stack.go new file mode 100644 index 0000000..a43897e --- /dev/null +++ b/internal/cli/stack.go @@ -0,0 +1,171 @@ +package cli + +import ( + "context" + "errors" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + "strconv" + "strings" + "text/tabwriter" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/stack" +) + +// proxyContainerPort is the port Traefik listens on inside siding-proxy. +const proxyContainerPort = 80 + +// existing is the stack of the current worktree, found through its labels. +type existing struct { + wt git.Worktree + id stack.Identity + project string +} + +func (s existing) ref(port int) stackRef { + return stackRef{Project: s.project, Repo: s.id.Repo, Branch: s.id.Branch, Worktree: s.id.Worktree, URL: address(s.id.Host(), port, "/")} +} + +// findStack returns the stack of the worktree containing the current directory: the project +// whose containers carry this worktree and branch. Asking Docker rather than recomputing the +// name keeps --repo, given once to up, out of every later command. +func (a *app) findStack(ctx context.Context) (existing, error) { + wt, err := a.opts.Git.Worktree(ctx, a.opts.Dir) + if err != nil { + return existing{}, err + } + if err := a.opts.Docker.Ping(ctx); err != nil { + return existing{}, err + } + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelWorktree: wt.Top, stack.LabelBranch: wt.Branch}) + if err != nil { + return existing{}, err + } + if len(cs) == 0 { + return existing{}, fail.New(fail.StackNotFound, "the worktree %s has no stack for the branch %s", wt.Top, wt.Branch). + WithHint("start it with siding up") + } + return existing{wt: wt, id: stack.FromLabels(cs[0].Labels), project: cs[0].Labels[stack.LabelProject]}, nil +} + +// proxyPort returns the host port of the running proxy. +func (a *app) proxyPort(ctx context.Context) (int, error) { + c, err := a.opts.Docker.Container(ctx, a.opts.Proxy.Container) + if err != nil { + return 0, err + } + if c == nil || c.State != "running" { + return 0, fail.New(fail.ProxyNotRunning, "%s is not running", a.opts.Proxy.Container). + WithHint("start it with siding setup") + } + for _, p := range c.Ports { + if p.ContainerPort == proxyContainerPort { + return p.HostPort, nil + } + } + return 0, fail.New(fail.ProxyNotRunning, "%s publishes no port", a.opts.Proxy.Container). + WithHint("recreate it with siding setup --port ") +} + +// shownPort is the proxy's port for addresses in ps, which also works while the proxy is down. +func (a *app) shownPort(ctx context.Context) int { + if port, err := a.proxyPort(ctx); err == nil { + return port + } + return a.opts.Proxy.DefaultPort +} + +// loadProject reads the compose configuration of a worktree as project. +func (a *app) loadProject(ctx context.Context, top, file, project string) (string, *compose.Project, error) { + path, err := compose.FindFile(a.opts.Dir, top, file) + if err != nil { + return "", nil, err + } + data, err := a.opts.Compose.Config(ctx, compose.Invocation{Dir: top, Project: project, Files: []string{path}}) + if err != nil { + return "", nil, err + } + p, err := compose.ParseConfig(data) + if err != nil { + return "", nil, fail.New(fail.ComposeFailed, "%v", err) + } + return path, p, nil +} + +// redactor redacts the values of the worktree's .env and of every service's environment. +func redactor(top string, p *compose.Project) (*stack.Redactor, error) { + var values []string + data, err := os.ReadFile(filepath.Join(top, ".env")) + switch { + case err == nil: + values = stack.DotEnvValues(string(data)) + case !errors.Is(err, fs.ErrNotExist): + return nil, fmt.Errorf("read .env: %w", err) + } + if p != nil { + for _, s := range p.Services { + for _, v := range s.Environment { + values = append(values, v) + } + } + } + return stack.NewRedactor(values...), nil +} + +// machineID returns dev.siding.machine for this machine. +func (a *app) machineID() (string, error) { + osID := "" + if a.opts.OSMachineID != nil { + osID = a.opts.OSMachineID() + } + return stack.MachineID(osID, a.opts.Home) +} + +// address is http://[:port]; port 80 is left out. +func address(host string, port int, path string) string { + if port == 80 || port == 0 { + return "http://" + host + path + } + return "http://" + host + ":" + strconv.Itoa(port) + path +} + +// byService groups containers by their Compose service. +func byService(cs []docker.Container) map[string][]docker.Container { + m := map[string][]docker.Container{} + for _, c := range cs { + s := c.Labels[stack.LabelService] + m[s] = append(m[s], c) + } + return m +} + +// table aligns rows into columns two spaces apart, each line starting with indent. +func table(indent string, rows [][]string) string { + var b strings.Builder + tw := tabwriter.NewWriter(&b, 0, 4, 2, ' ', 0) + for _, r := range rows { + _, _ = io.WriteString(tw, indent+strings.Join(r, "\t")+"\n") + } + _ = tw.Flush() + if b.Len() == 0 { + return "" + } + // tabwriter pads empty last columns with spaces. + lines := strings.Split(strings.TrimSuffix(b.String(), "\n"), "\n") + for i := range lines { + lines[i] = strings.TrimRight(lines[i], " ") + } + return strings.Join(lines, "\n") + "\n" +} + +// warnf prints a warning, on stderr. +func warnf(w io.Writer, format string, args ...any) { + _, _ = fmt.Fprintf(w, "warning: "+format+"\n", args...) +} diff --git a/internal/cli/status.go b/internal/cli/status.go new file mode 100644 index 0000000..88d822f --- /dev/null +++ b/internal/cli/status.go @@ -0,0 +1,229 @@ +package cli + +import ( + "context" + "fmt" + "io" + "strconv" + "strings" + "time" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/stack" + "github.com/tweinmann/siding/internal/verdict" +) + +const ( + // defaultTimeout is how long --wait waits by default. + defaultTimeout = 3 * time.Minute + // pollInterval is the time between two checks while waiting. + pollInterval = time.Second + // logTail is how many log lines a failure carries. + logTail = 20 +) + +type waitFlags struct { + wait bool + timeout time.Duration +} + +func (w *waitFlags) register(cmd *cobra.Command) { + cmd.Flags().BoolVar(&w.wait, "wait", false, "wait until the stack is Ready or Failed, at most --timeout") + cmd.Flags().DurationVar(&w.timeout, "timeout", defaultTimeout, "how long --wait waits") +} + +func newStatusCommand(a *app) *cobra.Command { + var asJSON bool + var file string + var w waitFlags + cmd := &cobra.Command{ + Use: "status", + Short: "Tell whether the worktree's stack runs: Ready, Starting, Failed or TimedOut", + Long: `Tell whether the worktree's stack runs. Ready means every service has a container, +every healthcheck is healthy, and every service with a route path answers through the proxy. +A failure names the service, the reason, a hint and its last log lines. + +Exit codes: 0 Ready, 1 Failed, 2 TimedOut (or Starting without --wait).`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + ctx := cmd.Context() + st, err := a.findStack(ctx) + if err != nil { + return err + } + port, err := a.proxyPort(ctx) + if err != nil { + return err + } + _, p, err := a.loadProject(ctx, st.wt.Top, file, st.project) + if err != nil { + return err + } + doc, err := a.status(ctx, st, p, port, w) + if err != nil { + return err + } + return printStatus(cmd.OutOrStdout(), doc, asJSON) + }, + } + cmd.Flags().StringVarP(&file, "file", "f", "", "compose file (default: compose.yaml or another name Compose looks for)") + w.register(cmd) + registerJSON(cmd, &asJSON) + return cmd +} + +// status evaluates the stack once, or until it is Ready or Failed or the time is up. +func (a *app) status(ctx context.Context, st existing, p *compose.Project, port int, w waitFlags) (statusDocument, error) { + routes, _, err := stack.Routes(p) + if err != nil { + return statusDocument{}, err + } + routed := map[string]verdict.Route{} + for _, r := range routes { + if r.Path != "" { + routed[r.Service] = verdict.Route{URL: address(st.id.Host(), port, r.Path), Port: r.Port} + } + } + start := a.opts.Clock.Now() + var base map[string]int + for { + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelProject: st.project}) + if err != nil { + return statusDocument{}, err + } + if base == nil { + base = map[string]int{} + for _, c := range cs { + base[c.ID] = c.RestartCount + } + } + answering := a.probe(ctx, st, routes, cs, port) + final := w.wait && a.opts.Clock.Now().Sub(start) >= w.timeout + res := verdict.Evaluate(verdict.Input{ + Services: p.ServiceNames(), Containers: cs, Routes: routed, Answering: answering, + RestartBase: base, Final: final, Timeout: w.timeout, + }) + if !w.wait || final || res.Verdict == verdict.Ready || res.Verdict == verdict.Failed { + return a.statusDocument(ctx, st, p, routes, port, cs, res) + } + if err := a.opts.Clock.Sleep(ctx, pollInterval); err != nil { + return statusDocument{}, err + } + } +} + +// probe asks the proxy, for every running service with a route path, whether it answers. +func (a *app) probe(ctx context.Context, st existing, routes []stack.Route, cs []docker.Container, port int) map[string]bool { + running := map[string]bool{} + for _, c := range cs { + if c.State == "running" && c.Health != "starting" && c.Health != "unhealthy" { + running[c.Labels[stack.LabelService]] = true + } + } + answering := map[string]bool{} + for _, r := range routes { + if r.Path == "" || !running[r.Service] { + continue + } + resp, err := a.opts.Probe.Get(ctx, "127.0.0.1:"+strconv.Itoa(port), st.id.Host(), r.Path) + answering[r.Service] = err == nil && resp.Answered(st.project+"/"+r.Service) + } + return answering +} + +func (a *app) statusDocument(ctx context.Context, st existing, p *compose.Project, routes []stack.Route, port int, + cs []docker.Container, res verdict.Result) (statusDocument, error) { + doc := statusDocument{header: newHeader(kindStatus), Verdict: res.Verdict, Stack: st.ref(port)} + for _, s := range res.Services { + ss := serviceStatus{Name: s.Name, State: s.State, Health: s.Health, Answering: s.Answering, Ready: s.Ready} + if r, ok := stack.Find(routes, s.Name); ok { + ss.Route = r.Path + ss.URL = address(st.id.ServiceHost(s.Name), port, "/") + } + doc.Services = append(doc.Services, ss) + } + if f := res.Failure; f != nil { + doc.Failure = &failureDocument{Service: f.Service, Reason: f.Reason, Hint: f.Hint} + if containers := byService(cs)[f.Service]; len(containers) > 0 { + lines, err := a.opts.Docker.Logs(ctx, failing(containers).ID, logTail) + if err != nil { + return statusDocument{}, err + } + r, err := redactor(st.wt.Top, p) + if err != nil { + return statusDocument{}, err + } + doc.Failure.Logs = r.Lines(lines) + } + } + return doc, nil +} + +// failing picks the container of a service whose logs explain its failure: one that is not +// running, else the first. +func failing(cs []docker.Container) docker.Container { + for _, c := range cs { + if c.State != "running" || c.Health == "unhealthy" { + return c + } + } + return cs[0] +} + +// printStatus prints the verdict and returns its exit code. +func printStatus(w io.Writer, doc statusDocument, asJSON bool) error { + if asJSON { + if err := printJSON(w, doc); err != nil { + return err + } + } else if err := printStatusText(w, doc); err != nil { + return err + } + switch doc.Verdict { + case verdict.Ready: + return nil + case verdict.Failed: + return exitCode(exitFailed) + default: + return exitCode(exitTimedOut) + } +} + +func printStatusText(w io.Writer, doc statusDocument) error { + var b strings.Builder + fmt.Fprintf(&b, "%s: %s %s\n", doc.Stack.Project, doc.Verdict, doc.Stack.URL) + var rows [][]string + for _, s := range doc.Services { + state := s.State + if s.Health != "" { + state += " (" + s.Health + ")" + } + reach := "" + if s.Answering != nil { + reach = "not answering" + if *s.Answering { + reach = "answering" + } + } + where := s.URL + if s.Route != "" { + where += " route " + s.Route + } + rows = append(rows, []string{s.Name, state, reach, where}) + } + b.WriteString(table(" ", rows)) + if f := doc.Failure; f != nil { + fmt.Fprintf(&b, "\n%s: %s, %s\n", f.Service, f.Reason, f.Hint) + if len(f.Logs) > 0 { + fmt.Fprintf(&b, "last log lines of %s:\n", f.Service) + for _, l := range f.Logs { + fmt.Fprintf(&b, " | %s\n", l) + } + } + } + _, err := io.WriteString(w, b.String()) + return err +} diff --git a/internal/cli/status_test.go b/internal/cli/status_test.go new file mode 100644 index 0000000..f6a357e --- /dev/null +++ b/internal/cli/status_test.go @@ -0,0 +1,175 @@ +package cli + +import ( + "encoding/json" + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/testutil" +) + +func TestStatus(t *testing.T) { + t.Parallel() + + t.Run("ready", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "mail", "web") + e.docker.add(container(e.id(), "db", "running", "healthy", 0)) + code, stdout, stderr := e.run("status", "--json") + if code != exitOK || stderr != "" { + t.Fatalf("exit %d, stderr %q, stdout %s", code, stderr, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "status-ready.json"), []byte(stdout)) + if !slices.Contains(e.probe.asked, "127.0.0.1:80 shop.main.localhost/api") { + t.Errorf("probes %v", e.probe.asked) + } + }) + + t.Run("failed with the log tail, redacted", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "db", "mail", "web") + api := container(e.id(), "api", "exited", "", 1) + e.docker.add(api) + e.docker.logs[api.ID] = []string{"starting", "connecting to postgres://app:not-a-real-token@db/app", "cannot connect to db"} + code, stdout, _ := e.run("status", "--json") + if code != exitFailed { + t.Fatalf("exit %d, stdout %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "status-failed.json"), []byte(stdout)) + if strings.Contains(stdout, "not-a-real-token") { + t.Error("a .env value reached the document") + } + }) + + t.Run("text of a failure", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "mail", "web") + db := container(e.id(), "db", "running", "unhealthy", 0) + db.HealthOutput = "pg_isready: no response" + e.docker.add(db) + e.docker.logs[db.ID] = []string{"FATAL: password authentication failed"} + code, stdout, _ := e.run("status") + want := `shop-main: Failed http://shop.main.localhost/ + api running answering http://api.shop.main.localhost/ route /api + db running (unhealthy) + mail running http://mail.shop.main.localhost/ + web running answering http://web.shop.main.localhost/ route / + +db: Unhealthy, its healthcheck fails: pg_isready: no response +last log lines of db: + | FATAL: password authentication failed +` + if code != exitFailed || stdout != want { + t.Errorf("exit %d, stdout:\n%s\nwant:\n%s", code, stdout, want) + } + }) + + t.Run("timed out", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "db", "mail", "web") + delete(e.probe.answers, "/") // web runs but never answers + code, stdout, _ := e.run("status", "--wait", "--timeout", "10s", "--json") + if code != exitTimedOut { + t.Fatalf("exit %d, stdout %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "status-timedout.json"), []byte(stdout)) + if e.clock.slept != 10 { + t.Errorf("slept %d times, want 10 one-second polls", e.clock.slept) + } + }) + + t.Run("waits until ready", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "mail", "web") + e.docker.add(container(e.id(), "db", "running", "starting", 0)) + e.clock.onSleep = func(n int) { + if n == 3 { + e.docker.update("db", func(c *docker.Container) { c.Health = "healthy" }) + } + } + code, stdout, _ := e.run("status", "--wait", "--json") + if code != exitOK || verdictOf(t, stdout) != "Ready" || e.clock.slept != 3 { + t.Errorf("exit %d, verdict %s, slept %d", code, verdictOf(t, stdout), e.clock.slept) + } + }) + + t.Run("starting without wait", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "mail", "web") + e.docker.add(container(e.id(), "db", "running", "starting", 0)) + code, stdout, _ := e.run("status", "--json") + if code != exitTimedOut || verdictOf(t, stdout) != "Starting" || e.clock.slept != 0 { + t.Errorf("exit %d, verdict %s", code, verdictOf(t, stdout)) + } + }) + + for _, tt := range []struct { + name string + setup func(e *env) + code string + }{ + {name: "no stack", setup: func(e *env) { e.proxyUp(80) }, code: fail.StackNotFound}, + {name: "no proxy", setup: func(e *env) { e.stackUp(e.id(), "web") }, code: fail.ProxyNotRunning}, + {name: "another branch's stack", setup: func(e *env) { + e.proxyUp(80) + other := e.id() + other.Branch = "feature/login" + e.stackUp(other, "web") + }, code: fail.StackNotFound}, + } { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + e := newEnv(t) + tt.setup(e) + if code, stdout, _ := e.run("status", "--json"); code != exitFailed || !strings.Contains(stdout, `"code": "`+tt.code+`"`) { + t.Errorf("exit %d, stdout %s", code, stdout) + } + }) + } +} + +func TestUpWait(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.compose.onRun = func(composeRun) { + e.stackUp(e.id(), "api", "db", "mail", "web") + } + code, stdout, stderr := e.run("up", "--wait", "--json") + if code != exitOK || verdictOf(t, stdout) != "Ready" || stderr != "" { + t.Errorf("exit %d, stdout %s, stderr %q", code, stdout, stderr) + } + + e.docker.update("api", func(c *docker.Container) { c.State, c.ExitCode = "exited", 1 }) + if code, stdout, _ := e.run("up", "--wait", "--json"); code != exitFailed || verdictOf(t, stdout) != "Failed" { + t.Errorf("after a crash: exit %d, verdict %s", code, verdictOf(t, stdout)) + } +} + +// verdictOf reads the verdict of one Status document, and fails the test unless stdout is +// exactly one document. +func verdictOf(t *testing.T, stdout string) string { + t.Helper() + var doc statusDocument + dec := json.NewDecoder(strings.NewReader(stdout)) + if err := dec.Decode(&doc); err != nil || dec.More() || doc.Kind != kindStatus { + t.Fatalf("not one Status document (%v): %s", err, stdout) + } + return doc.Verdict +} diff --git a/internal/cli/testdata/json/address.json b/internal/cli/testdata/json/address.json new file mode 100644 index 0000000..1bc1782 --- /dev/null +++ b/internal/cli/testdata/json/address.json @@ -0,0 +1,6 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Address", + "url": "http://api.shop.main.localhost/", + "service": "api" +} diff --git a/internal/cli/testdata/json/down-dryrun.json b/internal/cli/testdata/json/down-dryrun.json new file mode 100644 index 0000000..b775c4f --- /dev/null +++ b/internal/cli/testdata/json/down-dryrun.json @@ -0,0 +1,22 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Down", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "dryRun": true, + "containers": [ + "shop-main-db-1", + "shop-main-web-1" + ], + "networks": [ + "shop-main_default" + ], + "volumes": [ + "shop-main_data" + ] +} diff --git a/internal/cli/testdata/json/down.json b/internal/cli/testdata/json/down.json new file mode 100644 index 0000000..f67bdf8 --- /dev/null +++ b/internal/cli/testdata/json/down.json @@ -0,0 +1,18 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Down", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "containers": [ + "shop-main-db-1", + "shop-main-web-1" + ], + "networks": [ + "shop-main_default" + ] +} diff --git a/internal/cli/testdata/json/logs.json b/internal/cli/testdata/json/logs.json new file mode 100644 index 0000000..5d457cb --- /dev/null +++ b/internal/cli/testdata/json/logs.json @@ -0,0 +1,27 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Logs", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "lines": [ + "token *** accepted", + "three" + ] + }, + { + "name": "db", + "lines": [ + "listening on 5432", + "password ***" + ] + } + ] +} diff --git a/internal/cli/testdata/json/setup.json b/internal/cli/testdata/json/setup.json new file mode 100644 index 0000000..9c0f34c --- /dev/null +++ b/internal/cli/testdata/json/setup.json @@ -0,0 +1,10 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Setup", + "container": "siding-proxy", + "network": "siding", + "image": "traefik:v3.7.16", + "port": 80, + "url": "http://*.localhost/", + "state": "created" +} diff --git a/internal/cli/testdata/json/skill.json b/internal/cli/testdata/json/skill.json new file mode 100644 index 0000000..e5a492d --- /dev/null +++ b/internal/cli/testdata/json/skill.json @@ -0,0 +1,6 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Skill", + "name": "siding", + "text": "---\nname: siding\ndescription: Start this worktree's Docker Compose stack with siding and wait for the verdict, to check that a change to the app actually runs. Use after changing code, Dockerfiles, compose files or configuration that the running app depends on, and before reporting such work as done.\n---\n\n# Check that the app runs with siding\n\nsiding runs this git worktree's compose file as its own stack, reachable through a local proxy\nat `http://..localhost`, and tells you whether it actually runs. Every command\nbelow prints one JSON document with `--json`.\n\n## After a change that affects the running app\n\nRun:\n\n```sh\nsiding up --wait --json\n```\n\nIt starts or updates the stack and waits, three minutes at most, for one `Status` document.\nRead its `verdict` and the exit code:\n\n- **0, `Ready`**: every service runs, every healthcheck is healthy and every routed service\n answers through the proxy. Report the address in `stack.url` (and a service's `url` if it\n matters) together with your result.\n- **1, `Failed`**: `failure` names the `service`, the `reason` (`Exited`, `Unhealthy`,\n `Restarting`, `Missing`), a `hint` and the last log lines of that service in `logs`. Fix the\n cause in that service, then run `siding up --wait --json` again.\n- **2, `TimedOut`**: the stack was not ready in time. `failure` names the first service that\n was not ready: `NotAnswering` means it runs but does not answer through the proxy (often it\n listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady`\n means it is still starting. Read the hint and the logs; if it is only slow, run\n `siding status --wait --json --timeout 5m`.\n\nAn `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`,\n`message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`.\n\n## More detail\n\n- `siding status --json`: the current verdict, without waiting (exit 2 while starting).\n- `siding logs --json` or `siding logs --json --tail 200`: the last log lines per\n service. Values from `.env` and the services' environment appear as `***`.\n- `siding ps --json`: every stack on this machine, with state and memory.\n\n## Rules\n\n- Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file\n or the service labels `dev.siding.route` and `dev.siding.port` instead.\n- Leave stopping and removing stacks to the user; `siding down` only removes containers, and\n removing data is the user's decision.\n- Do not paste log lines into places outside this conversation.\n" +} diff --git a/internal/cli/testdata/json/stack.json b/internal/cli/testdata/json/stack.json new file mode 100644 index 0000000..33ac795 --- /dev/null +++ b/internal/cli/testdata/json/stack.json @@ -0,0 +1,30 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Stack", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "url": "http://api.shop.main.localhost/", + "route": "/api", + "port": 8080 + }, + { + "name": "mail", + "url": "http://mail.shop.main.localhost/", + "port": 8025 + }, + { + "name": "web", + "url": "http://web.shop.main.localhost/", + "route": "/", + "port": 3000 + } + ] +} diff --git a/internal/cli/testdata/json/stacklist-empty.json b/internal/cli/testdata/json/stacklist-empty.json new file mode 100644 index 0000000..e5eea1c --- /dev/null +++ b/internal/cli/testdata/json/stacklist-empty.json @@ -0,0 +1,4 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "StackList" +} diff --git a/internal/cli/testdata/json/stacklist.json b/internal/cli/testdata/json/stacklist.json new file mode 100644 index 0000000..8028677 --- /dev/null +++ b/internal/cli/testdata/json/stacklist.json @@ -0,0 +1,29 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "StackList", + "stacks": [ + { + "project": "shop-feature-login", + "repo": "shop", + "branch": "feature/login", + "worktree": "/src/shop-login", + "machine": "5c833a2dc2c206c2", + "agent": "claude", + "state": "partial", + "url": "http://shop.feature-login.localhost:8080/", + "memoryBytes": 524288000, + "worktreeMissing": true + }, + { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "machine": "5c833a2dc2c206c2", + "state": "running", + "url": "http://shop.main.localhost:8080/", + "memoryBytes": 314572800 + } + ], + "memoryBytes": 838860800 +} diff --git a/internal/cli/testdata/json/status-failed.json b/internal/cli/testdata/json/status-failed.json new file mode 100644 index 0000000..eef8687 --- /dev/null +++ b/internal/cli/testdata/json/status-failed.json @@ -0,0 +1,51 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Status", + "verdict": "Failed", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "state": "exited", + "route": "/api", + "url": "http://api.shop.main.localhost/", + "answering": false, + "ready": false + }, + { + "name": "db", + "state": "running", + "ready": true + }, + { + "name": "mail", + "state": "running", + "url": "http://mail.shop.main.localhost/", + "ready": true + }, + { + "name": "web", + "state": "running", + "route": "/", + "url": "http://web.shop.main.localhost/", + "answering": true, + "ready": true + } + ], + "failure": { + "service": "api", + "reason": "Exited", + "hint": "exited with code 1; see its last log lines", + "logs": [ + "starting", + "connecting to postgres://app:***@db/app", + "cannot connect to db" + ] + } +} diff --git a/internal/cli/testdata/json/status-ready.json b/internal/cli/testdata/json/status-ready.json new file mode 100644 index 0000000..cbe6652 --- /dev/null +++ b/internal/cli/testdata/json/status-ready.json @@ -0,0 +1,42 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Status", + "verdict": "Ready", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "state": "running", + "route": "/api", + "url": "http://api.shop.main.localhost/", + "answering": true, + "ready": true + }, + { + "name": "db", + "state": "running", + "health": "healthy", + "ready": true + }, + { + "name": "mail", + "state": "running", + "url": "http://mail.shop.main.localhost/", + "ready": true + }, + { + "name": "web", + "state": "running", + "route": "/", + "url": "http://web.shop.main.localhost/", + "answering": true, + "ready": true + } + ] +} diff --git a/internal/cli/testdata/json/status-timedout.json b/internal/cli/testdata/json/status-timedout.json new file mode 100644 index 0000000..aeb491c --- /dev/null +++ b/internal/cli/testdata/json/status-timedout.json @@ -0,0 +1,46 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Status", + "verdict": "TimedOut", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "state": "running", + "route": "/api", + "url": "http://api.shop.main.localhost/", + "answering": true, + "ready": true + }, + { + "name": "db", + "state": "running", + "ready": true + }, + { + "name": "mail", + "state": "running", + "url": "http://mail.shop.main.localhost/", + "ready": true + }, + { + "name": "web", + "state": "running", + "route": "/", + "url": "http://web.shop.main.localhost/", + "answering": false, + "ready": false + } + ], + "failure": { + "service": "web", + "reason": "NotAnswering", + "hint": "it runs, but http://shop.main.localhost/ does not answer through the proxy; check that it listens on port 3000 on 0.0.0.0, not only on 127.0.0.1" + } +} diff --git a/internal/cli/testdata/skill/SKILL.md b/internal/cli/testdata/skill/SKILL.md new file mode 100644 index 0000000..2f6d9c6 --- /dev/null +++ b/internal/cli/testdata/skill/SKILL.md @@ -0,0 +1,51 @@ +--- +name: siding +description: Start this worktree's Docker Compose stack with siding and wait for the verdict, to check that a change to the app actually runs. Use after changing code, Dockerfiles, compose files or configuration that the running app depends on, and before reporting such work as done. +--- + +# Check that the app runs with siding + +siding runs this git worktree's compose file as its own stack, reachable through a local proxy +at `http://..localhost`, and tells you whether it actually runs. Every command +below prints one JSON document with `--json`. + +## After a change that affects the running app + +Run: + +```sh +siding up --wait --json +``` + +It starts or updates the stack and waits, three minutes at most, for one `Status` document. +Read its `verdict` and the exit code: + +- **0, `Ready`**: every service runs, every healthcheck is healthy and every routed service + answers through the proxy. Report the address in `stack.url` (and a service's `url` if it + matters) together with your result. +- **1, `Failed`**: `failure` names the `service`, the `reason` (`Exited`, `Unhealthy`, + `Restarting`, `Missing`), a `hint` and the last log lines of that service in `logs`. Fix the + cause in that service, then run `siding up --wait --json` again. +- **2, `TimedOut`**: the stack was not ready in time. `failure` names the first service that + was not ready: `NotAnswering` means it runs but does not answer through the proxy (often it + listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady` + means it is still starting. Read the hint and the logs; if it is only slow, run + `siding status --wait --json --timeout 5m`. + +An `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`, +`message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`. + +## More detail + +- `siding status --json`: the current verdict, without waiting (exit 2 while starting). +- `siding logs --json` or `siding logs --json --tail 200`: the last log lines per + service. Values from `.env` and the services' environment appear as `***`. +- `siding ps --json`: every stack on this machine, with state and memory. + +## Rules + +- Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file + or the service labels `dev.siding.route` and `dev.siding.port` instead. +- Leave stopping and removing stacks to the user; `siding down` only removes containers, and + removing data is the user's decision. +- Do not paste log lines into places outside this conversation. diff --git a/internal/cli/up.go b/internal/cli/up.go new file mode 100644 index 0000000..0b4ac85 --- /dev/null +++ b/internal/cli/up.go @@ -0,0 +1,173 @@ +package cli + +import ( + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" +) + +func newUpCommand(a *app) *cobra.Command { + var asJSON bool + var f upFlags + var w waitFlags + cmd := &cobra.Command{ + Use: "up [-- ]", + Short: "Start the worktree's stack from its unchanged compose file", + Long: `Start the worktree's stack: the compose file as the Compose project -, +plus .siding/override.yml, which resets every published port and routes the HTTP services +through siding-proxy. The compose file is never edited. + +Arguments after -- go to docker compose up, e.g. siding up -- --build web. +With --wait, up ends like siding status --wait, with its verdict and exit code.`, + Args: func(cmd *cobra.Command, args []string) error { + if n := cmd.ArgsLenAtDash(); n > 0 || (n < 0 && len(args) > 0) { + return fail.New(fail.InvalidArguments, "siding up takes no arguments before --, got %q", args[0]). + WithHint("pass docker compose arguments after --, e.g. siding up -- --build") + } + return nil + }, + RunE: func(cmd *cobra.Command, args []string) error { + if n := cmd.ArgsLenAtDash(); n >= 0 { + f.compose = args[n:] + } + if f.agent == "" && a.opts.Getenv != nil { + f.agent = a.opts.Getenv("SIDING_AGENT") + } + return a.up(cmd, f, w, asJSON) + }, + } + cmd.Flags().StringVarP(&f.file, "file", "f", "", "compose file (default: compose.yaml or another name Compose looks for)") + cmd.Flags().StringVar(&f.repo, "repo", "", "repository name (default: the main worktree's directory name)") + cmd.Flags().StringVar(&f.agent, "agent", "", "agent name for the label dev.siding.agent (default: $SIDING_AGENT)") + w.register(cmd) + registerJSON(cmd, &asJSON) + return cmd +} + +type upFlags struct { + file, repo, agent string + compose []string +} + +func (a *app) up(cmd *cobra.Command, f upFlags, w waitFlags, asJSON bool) error { + ctx := cmd.Context() + stderr := cmd.ErrOrStderr() + wt, err := a.opts.Git.Worktree(ctx, a.opts.Dir) + if err != nil { + return err + } + machine, err := a.machineID() + if err != nil { + return err + } + repo := f.repo + if repo == "" { + repo = wt.RepoName() + } + id := stack.Identity{Repo: repo, Branch: wt.Branch, Worktree: wt.Top, Machine: machine, Agent: f.agent} + project := id.Project() + + if err := a.opts.Docker.Ping(ctx); err != nil { + return err + } + port, err := a.proxyPort(ctx) + if err != nil { + return err + } + existingContainers, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelProject: project}) + if err != nil { + return err + } + for _, c := range existingContainers { + other := stack.FromLabels(c.Labels) + // A plain Compose project without siding's labels is adopted. + if other.Worktree == "" { + continue + } + if other.Worktree != id.Worktree || other.Branch != id.Branch { + return fail.New(fail.SlugCollision, "the Compose project %s already belongs to the branch %s in %s", project, other.Branch, other.Worktree). + WithHint("the branches %q and %q have the same slug; rename one, or pass --repo to tell them apart", other.Branch, id.Branch) + } + } + + file, p, err := a.loadProject(ctx, wt.Top, f.file, project) + if err != nil { + return err + } + routes, warnings, err := stack.Routes(p) + if err != nil { + return err + } + for _, w := range warnings { + warnf(stderr, "%s", w) + } + override, err := writeOverride(wt.Top, stack.Override(p, id, routes, a.opts.Proxy.Network)) + if err != nil { + return err + } + + inv := compose.Invocation{Dir: wt.Top, Project: project, Files: []string{file, override}} + args := append([]string{"up", "-d", "--remove-orphans"}, f.compose...) + // Compose's progress, and anything it prints, goes to stderr: stdout is for siding's result. + if err := a.opts.Compose.Run(ctx, inv, args, stderr, stderr); err != nil { + return err + } + + st := existing{wt: wt, id: id, project: project} + if w.wait { + doc, err := a.status(ctx, st, p, port, w) + if err != nil { + return err + } + return printStatus(cmd.OutOrStdout(), doc, asJSON) + } + doc := stackDocument{header: newHeader(kindStack), Stack: st.ref(port)} + for _, r := range routes { + doc.Services = append(doc.Services, serviceAddress{Name: r.Service, URL: address(id.ServiceHost(r.Service), port, "/"), Route: r.Path, Port: r.Port}) + } + if asJSON { + return printJSON(cmd.OutOrStdout(), doc) + } + return printStackText(cmd, doc) +} + +// writeOverride writes .siding/override.yml and .siding/.gitignore, which ignores the whole +// directory, itself included, so .siding/ never makes the worktree dirty. +func writeOverride(top string, content []byte) (string, error) { + path := filepath.Join(top, stack.OverrideFile) + dir := filepath.Dir(path) + if err := os.MkdirAll(dir, 0o755); err != nil { + return "", fmt.Errorf("write %s: %w", stack.OverrideFile, err) + } + if err := os.WriteFile(filepath.Join(dir, ".gitignore"), []byte("# Written by siding: nothing in .siding/ belongs in git.\n*\n"), 0o644); err != nil { + return "", fmt.Errorf("write .siding/.gitignore: %w", err) + } + if err := os.WriteFile(path, content, 0o644); err != nil { + return "", fmt.Errorf("write %s: %w", stack.OverrideFile, err) + } + return path, nil +} + +func printStackText(cmd *cobra.Command, doc stackDocument) error { + var b strings.Builder + fmt.Fprintf(&b, "%s is up at %s\n", doc.Stack.Project, doc.Stack.URL) + var rows [][]string + for _, s := range doc.Services { + route := "" + if s.Route != "" { + route = "route " + s.Route + } + rows = append(rows, []string{s.Name, s.URL, route}) + } + b.WriteString(table(" ", rows)) + b.WriteString("Wait for the verdict with: siding status --wait\n") + _, err := fmt.Fprint(cmd.OutOrStdout(), b.String()) + return err +} diff --git a/internal/cli/up_test.go b/internal/cli/up_test.go new file mode 100644 index 0000000..abe7133 --- /dev/null +++ b/internal/cli/up_test.go @@ -0,0 +1,163 @@ +package cli + +import ( + "os" + "os/exec" + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/testutil" +) + +func TestUp(t *testing.T) { + t.Parallel() + + t.Run("starts the stack", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + before, _ := os.ReadFile(filepath.Join(e.repo, "compose.yaml")) + code, stdout, stderr := e.run("up", "--json", "--", "--build", "web") + if code != exitOK || stderr != "" { + t.Fatalf("exit %d, stderr %q", code, stderr) + } + testutil.Golden(t, filepath.Join("testdata", "json", "stack.json"), []byte(stdout)) + + runs := e.compose.recorded() + if len(runs) != 1 { + t.Fatalf("compose runs %+v", runs) + } + r := runs[0] + wantFiles := []string{filepath.Join(e.repo, "compose.yaml"), filepath.Join(e.repo, ".siding", "override.yml")} + if r.inv.Project != "shop-main" || !slices.Equal(r.inv.Files, wantFiles) || + !slices.Equal(r.args, []string{"up", "-d", "--remove-orphans", "--build", "web"}) { + t.Errorf("compose ran %+v", r) + } + override, err := os.ReadFile(wantFiles[1]) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(override), "ports: !reset []") || !strings.Contains(string(override), "Host(`shop.main.localhost`) && PathPrefix(`/api`)") { + t.Errorf("override:\n%s", override) + } + after, _ := os.ReadFile(filepath.Join(e.repo, "compose.yaml")) + if string(after) != string(before) { + t.Error("the compose file changed") + } + if out, err := exec.Command("git", "-C", e.repo, "status", "--porcelain").Output(); err != nil || len(out) != 0 { + t.Errorf("the worktree is dirty after up: %q, %v", out, err) + } + }) + + t.Run("text", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(8080) + code, stdout, _ := e.run("up") + want := `shop-main is up at http://shop.main.localhost:8080/ + api http://api.shop.main.localhost:8080/ route /api + mail http://mail.shop.main.localhost:8080/ + web http://web.shop.main.localhost:8080/ route / +Wait for the verdict with: siding status --wait +` + if code != exitOK || stdout != want { + t.Errorf("exit %d, stdout:\n%s\nwant:\n%s", code, stdout, want) + } + }) + + t.Run("agent", func(t *testing.T) { + t.Parallel() + for _, tt := range []struct { + name, env, flag, want string + }{ + {name: "from SIDING_AGENT", env: "claude", want: `"dev.siding.agent": "claude"`}, + {name: "the flag wins", env: "claude", flag: "codex", want: `"dev.siding.agent": "codex"`}, + {name: "none", want: ""}, + } { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.envVars["SIDING_AGENT"] = tt.env + args := []string{"up"} + if tt.flag != "" { + args = append(args, "--agent", tt.flag) + } + if code, _, stderr := e.run(args...); code != exitOK { + t.Fatalf("exit %d: %s", code, stderr) + } + override, _ := os.ReadFile(filepath.Join(e.repo, ".siding", "override.yml")) + got := strings.Contains(string(override), "dev.siding.agent") + if (tt.want == "") == got || (tt.want != "" && !strings.Contains(string(override), tt.want)) { + t.Errorf("override has agent %v, want %q", got, tt.want) + } + }) + } + }) + + t.Run("repo override", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + if code, stdout, _ := e.run("up", "--repo", "store"); code != exitOK || !strings.HasPrefix(stdout, "store-main is up at http://store.main.localhost/") { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) + + failures := []struct { + name string + setup func(e *env) + code string + }{ + {name: "proxy not running", setup: func(*env) {}, code: fail.ProxyNotRunning}, + {name: "outside a repository", setup: func(e *env) { e.proxyUp(80); e.opts.Dir = t.TempDir() }, code: fail.NotAGitRepository}, + {name: "detached HEAD", setup: func(e *env) { e.proxyUp(80); e.git("checkout", "-q", "--detach") }, code: fail.NoBranch}, + {name: "no compose file", setup: func(e *env) { + e.proxyUp(80) + _ = os.Remove(filepath.Join(e.repo, "compose.yaml")) + }, code: fail.ComposeFileNotFound}, + {name: "slug collision", setup: func(e *env) { + e.proxyUp(80) + other := e.id() + other.Branch, other.Worktree = "main", "/src/elsewhere" + e.docker.add(container(other, "web", "running", "", 0)) + }, code: fail.SlugCollision}, + {name: "compose fails", setup: func(e *env) { + e.proxyUp(80) + e.compose.runErr = fail.New(fail.ComposeFailed, "docker compose up -d failed: pull access denied for nope") + }, code: fail.ComposeFailed}, + {name: "two services on one path", setup: func(e *env) { + e.proxyUp(80) + e.compose.config = `{"services": {"a": {"expose": ["80"], "labels": {"dev.siding.route": "/"}}, "b": {"expose": ["80"], "labels": {"dev.siding.route": "/"}}}}` + }, code: fail.InvalidRoute}, + } + for _, tt := range failures { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + e := newEnv(t) + tt.setup(e) + code, stdout, _ := e.run("up", "--json") + if code != exitFailed || !strings.Contains(stdout, `"code": "`+tt.code+`"`) { + t.Errorf("exit %d, stdout %s; want code %s", code, stdout, tt.code) + } + if tt.code != fail.ComposeFailed && len(e.compose.recorded()) != 0 { + t.Errorf("compose ran although up failed: %+v", e.compose.recorded()) + } + }) + } + + t.Run("a plain Compose project of the same name is adopted", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + plain := container(e.id(), "web", "running", "", 0) + plain.Labels = map[string]string{"com.docker.compose.project": "shop-main", "com.docker.compose.service": "web"} + e.docker.add(plain) + if code, _, stderr := e.run("up"); code != exitOK { + t.Errorf("exit %d: %s", code, stderr) + } + }) +} diff --git a/internal/compose/compose.go b/internal/compose/compose.go new file mode 100644 index 0000000..043092c --- /dev/null +++ b/internal/compose/compose.go @@ -0,0 +1,165 @@ +package compose + +import ( + "bytes" + "context" + "errors" + "fmt" + "io" + "os/exec" + "slices" + "strings" + "sync" + + "github.com/tweinmann/siding/internal/fail" +) + +// MinVersion is the oldest Compose siding works with: `ports: !reset []` needs 2.24. +const MinVersion = "2.24" + +// Invocation names a Compose project and the files it is made of. +type Invocation struct { + // Dir is the working directory; Compose's project directory is the first file's. + Dir string + Project string + // Files are passed as -f, in order. Without files, Compose acts on the project by name. + Files []string +} + +func (inv Invocation) args(rest ...string) []string { + args := []string{"compose", "-p", inv.Project} + for _, f := range inv.Files { + args = append(args, "-f", f) + } + return append(args, rest...) +} + +// CLI runs docker compose. Its zero value uses "docker" from PATH. +type CLI struct { + // Binary is the docker executable; empty means "docker". + Binary string + + once sync.Once + versionErr error +} + +func (c *CLI) bin() string { + if c.Binary == "" { + return "docker" + } + return c.Binary +} + +// checkVersion fails with ComposeFailed unless Compose is installed and at least MinVersion. +// It runs once per CLI. +func (c *CLI) checkVersion(ctx context.Context) error { + c.once.Do(func() { + out, err := exec.CommandContext(ctx, c.bin(), "compose", "version", "--short").Output() + if err != nil { + c.versionErr = fail.New(fail.ComposeFailed, "docker compose is not available: %v", err). + WithHint("install the Docker Compose plugin %s or later", MinVersion) + return + } + v := strings.TrimPrefix(strings.TrimSpace(string(out)), "v") + if !AtLeast(v, MinVersion) { + c.versionErr = fail.New(fail.ComposeFailed, "docker compose %s is too old", v). + WithHint("siding needs Docker Compose %s or later", MinVersion) + } + }) + return c.versionErr +} + +// AtLeast compares dotted version numbers: AtLeast("2.24.1", "2.24") is true. +func AtLeast(have, want string) bool { + h, w := strings.Split(have, "."), strings.Split(want, ".") + for i := range w { + var hn, wn int + if i < len(h) { + _, _ = fmt.Sscanf(h[i], "%d", &hn) + } + _, _ = fmt.Sscanf(w[i], "%d", &wn) + if hn != wn { + return hn > wn + } + } + return true +} + +// Config returns the output of docker compose config --format json. +func (c *CLI) Config(ctx context.Context, inv Invocation) ([]byte, error) { + if err := c.checkVersion(ctx); err != nil { + return nil, err + } + var out, errOut bytes.Buffer + if err := c.exec(ctx, inv, inv.args("config", "--format", "json"), &out, &errOut); err != nil { + return nil, composeError("read the compose configuration", errOut.String(), err) + } + return out.Bytes(), nil +} + +// Run runs docker compose with args, streaming its output. On failure the error carries the +// last lines Compose printed on stderr. +func (c *CLI) Run(ctx context.Context, inv Invocation, args []string, stdout, stderr io.Writer) error { + if err := c.checkVersion(ctx); err != nil { + return err + } + tail := &tailBuffer{max: 20} + if err := c.exec(ctx, inv, inv.args(args...), stdout, io.MultiWriter(stderr, tail)); err != nil { + if ctx.Err() != nil { + return ctx.Err() // interrupted, e.g. Ctrl-C during logs --follow + } + return composeError("docker compose "+strings.Join(args, " "), tail.String(), err) + } + return nil +} + +func (c *CLI) exec(ctx context.Context, inv Invocation, args []string, stdout, stderr io.Writer) error { + cmd := exec.CommandContext(ctx, c.bin(), args...) + cmd.Dir = inv.Dir + // No stdin: a question from Compose fails instead of waiting for an answer nobody gives. + cmd.Stdin = nil + cmd.Stdout = stdout + cmd.Stderr = stderr + return cmd.Run() +} + +func composeError(what, output string, err error) error { + var exit *exec.ExitError + if !errors.As(err, &exit) { + return fail.New(fail.ComposeFailed, "%s: %v", what, err) + } + output = strings.TrimSpace(output) + if output == "" { + return fail.New(fail.ComposeFailed, "%s failed with exit code %d", what, exit.ExitCode()) + } + return fail.New(fail.ComposeFailed, "%s failed: %s", what, output) +} + +// tailBuffer keeps the last max lines written to it. +type tailBuffer struct { + max int + lines []string + partial string +} + +func (t *tailBuffer) Write(p []byte) (int, error) { + s := t.partial + string(p) + parts := strings.Split(s, "\n") + t.partial = parts[len(parts)-1] + t.lines = append(t.lines, parts[:len(parts)-1]...) + if len(t.lines) > t.max { + t.lines = t.lines[len(t.lines)-t.max:] + } + return len(p), nil +} + +func (t *tailBuffer) String() string { + lines := slices.Clone(t.lines) + if t.partial != "" { + lines = append(lines, t.partial) + } + if len(lines) > t.max { + lines = lines[len(lines)-t.max:] + } + return strings.Join(lines, "\n") +} diff --git a/internal/compose/config.go b/internal/compose/config.go new file mode 100644 index 0000000..753f6b5 --- /dev/null +++ b/internal/compose/config.go @@ -0,0 +1,188 @@ +// Package compose runs docker compose and reads the normalised configuration it prints. siding +// never parses a compose file itself: interpolation, profiles, include and extends are +// Compose's business. +package compose + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "slices" + "sort" + "strconv" + "strings" + + "github.com/tweinmann/siding/internal/fail" +) + +// FileNames are the compose files Compose itself looks for, in its order. +var FileNames = []string{"compose.yaml", "compose.yml", "docker-compose.yaml", "docker-compose.yml"} + +// FindFile returns the compose file of a worktree: file when given (relative to dir), else the +// first of FileNames in top. +func FindFile(dir, top, file string) (string, error) { + if file != "" { + if !filepath.IsAbs(file) { + file = filepath.Join(dir, file) + } + if _, err := os.Stat(file); err != nil { + return "", fail.New(fail.ComposeFileNotFound, "compose file %s not found", file) + } + return file, nil + } + for _, name := range FileNames { + path := filepath.Join(top, name) + if _, err := os.Stat(path); err == nil { + return path, nil + } + } + return "", fail.New(fail.ComposeFileNotFound, "no compose file in %s", top). + WithHint("siding looks for %s; name another one with --file", strings.Join(FileNames, ", ")) +} + +// Project is the part of a normalised compose configuration siding needs. +type Project struct { + Name string + Services map[string]Service + Networks map[string]Resource + Volumes map[string]Resource +} + +// ServiceNames returns the service names in the order Compose prints them (sorted). +func (p *Project) ServiceNames() []string { + names := make([]string, 0, len(p.Services)) + for n := range p.Services { + names = append(names, n) + } + sort.Strings(names) + return names +} + +// Service is one service of a Project. +type Service struct { + // Ports are the distinct container ports from ports (targets) and expose, ascending. + Ports []int + Labels map[string]string + Networks []string // keys of the service's networks, sorted + NetworkMode string + Environment map[string]string +} + +// Resource is a top-level network or volume. +type Resource struct { + Name string + External bool +} + +type rawProject struct { + Name string `json:"name"` + Services map[string]rawService `json:"services"` + Networks map[string]rawResource `json:"networks"` + Volumes map[string]rawResource `json:"volumes"` +} + +type rawService struct { + Ports []struct { + Target int `json:"target"` + Protocol string `json:"protocol"` + } `json:"ports"` + Expose []json.RawMessage `json:"expose"` + Labels map[string]string `json:"labels"` + Networks map[string]json.RawMessage `json:"networks"` + NetworkMode string `json:"network_mode"` + Environment map[string]*json.RawMessage `json:"environment"` +} + +type rawResource struct { + Name string `json:"name"` + External bool `json:"external"` +} + +// ParseConfig reads the output of docker compose config --format json. +func ParseConfig(data []byte) (*Project, error) { + var raw rawProject + if err := json.Unmarshal(data, &raw); err != nil { + return nil, fmt.Errorf("read compose config: %w", err) + } + p := &Project{Name: raw.Name, Services: map[string]Service{}, Networks: map[string]Resource{}, Volumes: map[string]Resource{}} + for name, rs := range raw.Services { + s := Service{Labels: rs.Labels, NetworkMode: rs.NetworkMode, Environment: map[string]string{}} + if s.Labels == nil { + s.Labels = map[string]string{} + } + ports := map[int]bool{} + for _, port := range rs.Ports { + if port.Protocol == "" || port.Protocol == "tcp" { + ports[port.Target] = true + } + } + for _, e := range rs.Expose { + expanded, err := parseExpose(e) + if err != nil { + return nil, fmt.Errorf("service %s: %w", name, err) + } + for _, n := range expanded { + ports[n] = true + } + } + for n := range ports { + s.Ports = append(s.Ports, n) + } + slices.Sort(s.Ports) + for n := range rs.Networks { + s.Networks = append(s.Networks, n) + } + slices.Sort(s.Networks) + for k, v := range rs.Environment { + if v == nil { + continue + } + var str string + if err := json.Unmarshal(*v, &str); err == nil { + s.Environment[k] = str + } + } + p.Services[name] = s + } + for k, r := range raw.Networks { + p.Networks[k] = Resource(r) + } + for k, r := range raw.Volumes { + p.Volumes[k] = Resource(r) + } + return p, nil +} + +// parseExpose expands one expose entry ("3000", "3000/tcp", "9000-9001", or a number) into +// TCP ports. +func parseExpose(raw json.RawMessage) ([]int, error) { + var s string + if err := json.Unmarshal(raw, &s); err != nil { + var n int + if err := json.Unmarshal(raw, &n); err != nil { + return nil, fmt.Errorf("expose entry %s", raw) + } + return []int{n}, nil + } + spec, proto, _ := strings.Cut(s, "/") + if proto != "" && proto != "tcp" { + return nil, nil + } + lo, hi, isRange := strings.Cut(spec, "-") + first, err := strconv.Atoi(lo) + if err != nil { + return nil, fmt.Errorf("expose entry %q", s) + } + last := first + if isRange { + if last, err = strconv.Atoi(hi); err != nil || last < first { + return nil, fmt.Errorf("expose entry %q", s) + } + } + // Only the number of ports matters (one makes an HTTP service), so a range is its ends. + if last > first { + return []int{first, last}, nil + } + return []int{first}, nil +} diff --git a/internal/compose/config_test.go b/internal/compose/config_test.go new file mode 100644 index 0000000..68152f1 --- /dev/null +++ b/internal/compose/config_test.go @@ -0,0 +1,126 @@ +package compose + +import ( + "errors" + "os" + "path/filepath" + "reflect" + "testing" + + "github.com/tweinmann/siding/internal/fail" +) + +// testdata/routing/config.json is what `docker compose -p shop-main config --format json` +// prints for testdata/routing/compose.yaml with the Compose pinned in the devcontainer. After a +// Compose upgrade, capture it again in that directory. +func TestParseConfig(t *testing.T) { + t.Parallel() + data, err := os.ReadFile(filepath.Join("testdata", "routing", "config.json")) + if err != nil { + t.Fatal(err) + } + p, err := ParseConfig(data) + if err != nil { + t.Fatal(err) + } + if p.Name != "shop-main" { + t.Errorf("name %q", p.Name) + } + if got, want := p.ServiceNames(), []string{"admin", "api", "db", "mail", "tools", "web", "worker"}; !reflect.DeepEqual(got, want) { + t.Errorf("services %v, want %v", got, want) + } + services := []struct { + name string + ports []int + networks []string + networkMode string + route string + }{ + {name: "web", ports: []int{3000}, networks: []string{"default"}, route: "/"}, + {name: "api", ports: []int{8080, 9229}, networks: []string{"backend", "default"}, route: "/api"}, + {name: "mail", ports: []int{8025}, networks: []string{"default"}}, + {name: "admin", ports: []int{80}, networks: []string{"default"}, route: "false"}, + {name: "db", ports: []int{5432}, networks: []string{"backend"}}, + {name: "worker", networks: []string{"default"}}, + {name: "tools", ports: []int{7000, 7001}, networkMode: "host"}, + } + for _, want := range services { + s := p.Services[want.name] + if !reflect.DeepEqual(s.Ports, want.ports) || !reflect.DeepEqual(s.Networks, want.networks) || + s.NetworkMode != want.networkMode || s.Labels["dev.siding.route"] != want.route { + t.Errorf("%s: ports %v, networks %v, network_mode %q, route %q; want %v, %v, %q, %q", want.name, + s.Ports, s.Networks, s.NetworkMode, s.Labels["dev.siding.route"], want.ports, want.networks, want.networkMode, want.route) + } + } + if got := p.Services["api"].Environment["API_TOKEN"]; got != "not-a-real-api-token" { + t.Errorf("env_file value %q, want it inlined into environment", got) + } + if got := p.Services["web"].Environment["DB_URL"]; got != "postgres://app:not-a-real-token@db/app" { + t.Errorf("interpolated value %q", got) + } + if got := p.Volumes["shared"]; !got.External || got.Name != "shared-cache" { + t.Errorf("external volume %+v", got) + } + if got := p.Volumes["data"]; got.External || got.Name != "shop-main_data" { + t.Errorf("volume %+v", got) + } + if got := p.Networks["backend"]; got.External || got.Name != "shop-main_backend" { + t.Errorf("network %+v", got) + } +} + +func TestFindFile(t *testing.T) { + t.Parallel() + top := t.TempDir() + if _, err := FindFile(top, top, ""); codeOf(err) != fail.ComposeFileNotFound { + t.Fatalf("empty worktree: %v", err) + } + for _, name := range []string{"docker-compose.yml", "compose.yaml"} { + if err := os.WriteFile(filepath.Join(top, name), nil, 0o644); err != nil { + t.Fatal(err) + } + } + if got, err := FindFile(top, top, ""); err != nil || got != filepath.Join(top, "compose.yaml") { + t.Errorf("got %q, %v; want compose.yaml first", got, err) + } + sub := filepath.Join(top, "sub") + if err := os.MkdirAll(sub, 0o755); err != nil { + t.Fatal(err) + } + if got, err := FindFile(sub, top, "../docker-compose.yml"); err != nil || got != filepath.Join(top, "docker-compose.yml") { + t.Errorf("--file relative to the current directory: %q, %v", got, err) + } + if _, err := FindFile(top, top, "nope.yml"); codeOf(err) != fail.ComposeFileNotFound { + t.Errorf("missing --file: %v", err) + } +} + +func TestAtLeast(t *testing.T) { + t.Parallel() + for _, tt := range []struct { + have string + want bool + }{{"2.24.0", true}, {"2.24", true}, {"2.23.9", false}, {"5.6.1", true}, {"2.3", false}, {"2.100", true}} { + if got := AtLeast(tt.have, MinVersion); got != tt.want { + t.Errorf("AtLeast(%q, %q) = %v, want %v", tt.have, MinVersion, got, tt.want) + } + } +} + +func TestTailBuffer(t *testing.T) { + t.Parallel() + b := &tailBuffer{max: 2} + _, _ = b.Write([]byte("one\ntwo\nthr")) + _, _ = b.Write([]byte("ee\nfour")) + if got := b.String(); got != "three\nfour" { + t.Errorf("got %q", got) + } +} + +func codeOf(err error) string { + var e *fail.Error + if errors.As(err, &e) { + return e.Code + } + return "" +} diff --git a/internal/compose/testdata/routing/.env b/internal/compose/testdata/routing/.env new file mode 100644 index 0000000..c9d92b7 --- /dev/null +++ b/internal/compose/testdata/routing/.env @@ -0,0 +1 @@ +DB_PASSWORD=not-a-real-token diff --git a/internal/compose/testdata/routing/api.env b/internal/compose/testdata/routing/api.env new file mode 100644 index 0000000..a17f2ca --- /dev/null +++ b/internal/compose/testdata/routing/api.env @@ -0,0 +1 @@ +API_TOKEN=not-a-real-api-token diff --git a/internal/compose/testdata/routing/compose.yaml b/internal/compose/testdata/routing/compose.yaml new file mode 100644 index 0000000..466b118 --- /dev/null +++ b/internal/compose/testdata/routing/compose.yaml @@ -0,0 +1,40 @@ +# Every case the parser and the routing table must handle. +services: + web: + image: busybox:1.37.0 + ports: ["3000:3000"] + labels: [dev.siding.route=/] + healthcheck: {test: ["CMD", "true"]} + environment: + NODE_ENV: development + DB_URL: postgres://app:${DB_PASSWORD}@db/app + api: + image: busybox:1.37.0 + expose: ["8080", "9229"] + env_file: api.env + networks: [backend, default] + labels: {dev.siding.route: /api, dev.siding.port: "8080"} + mail: + image: busybox:1.37.0 + expose: ["8025/tcp", "1025/udp"] + admin: + image: busybox:1.37.0 + ports: ["127.0.0.1:9000:80"] + labels: {dev.siding.route: "false"} + db: + image: busybox:1.37.0 + ports: ["5432:5432"] + volumes: [data:/var/lib/data, shared:/shared] + networks: + backend: {aliases: [database]} + worker: + image: busybox:1.37.0 + tools: + image: busybox:1.37.0 + network_mode: host + expose: ["7000-7001"] +networks: + backend: {} +volumes: + data: {} + shared: {external: true, name: shared-cache} diff --git a/internal/compose/testdata/routing/config.json b/internal/compose/testdata/routing/config.json new file mode 100644 index 0000000..bcff27a --- /dev/null +++ b/internal/compose/testdata/routing/config.json @@ -0,0 +1,156 @@ +{ + "name": "shop-main", + "networks": { + "backend": { + "name": "shop-main_backend", + "ipam": {} + }, + "default": { + "name": "shop-main_default", + "ipam": {} + } + }, + "services": { + "admin": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "labels": { + "dev.siding.route": "false" + }, + "networks": { + "default": null + }, + "ports": [ + { + "mode": "ingress", + "host_ip": "127.0.0.1", + "target": 80, + "published": "9000", + "protocol": "tcp" + } + ] + }, + "api": { + "command": null, + "entrypoint": null, + "environment": { + "API_TOKEN": "not-a-real-api-token" + }, + "image": "busybox:1.37.0", + "labels": { + "dev.siding.port": "8080", + "dev.siding.route": "/api" + }, + "networks": { + "backend": null, + "default": null + }, + "expose": [ + "8080", + "9229" + ] + }, + "db": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "networks": { + "backend": { + "aliases": [ + "database" + ] + } + }, + "volumes": [ + { + "type": "volume", + "source": "data", + "target": "/var/lib/data", + "volume": {} + }, + { + "type": "volume", + "source": "shared", + "target": "/shared", + "volume": {} + } + ], + "ports": [ + { + "mode": "ingress", + "target": 5432, + "published": "5432", + "protocol": "tcp" + } + ] + }, + "mail": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "networks": { + "default": null + }, + "expose": [ + "8025/tcp", + "1025/udp" + ] + }, + "tools": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "network_mode": "host", + "expose": [ + "7000-7001" + ] + }, + "web": { + "command": null, + "entrypoint": null, + "environment": { + "DB_URL": "postgres://app:not-a-real-token@db/app", + "NODE_ENV": "development" + }, + "image": "busybox:1.37.0", + "labels": { + "dev.siding.route": "/" + }, + "networks": { + "default": null + }, + "healthcheck": { + "test": [ + "CMD", + "true" + ] + }, + "ports": [ + { + "mode": "ingress", + "target": 3000, + "published": "3000", + "protocol": "tcp" + } + ] + }, + "worker": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "networks": { + "default": null + } + } + }, + "volumes": { + "data": { + "name": "shop-main_data" + }, + "shared": { + "name": "shared-cache", + "external": true + } + } +} \ No newline at end of file diff --git a/internal/docker/docker.go b/internal/docker/docker.go new file mode 100644 index 0000000..26b192d --- /dev/null +++ b/internal/docker/docker.go @@ -0,0 +1,324 @@ +// Package docker reads the state of stacks from the Docker Engine API. Docker is siding's +// database: containers, networks and volumes with their labels are all it knows. +package docker + +import ( + "bufio" + "bytes" + "context" + "encoding/json" + "fmt" + "os" + "os/exec" + "strconv" + "strings" + "sync" + "time" + + cerrdefs "github.com/containerd/errdefs" + "github.com/moby/moby/api/pkg/stdcopy" + "github.com/moby/moby/api/types/container" + "github.com/moby/moby/client" + + "github.com/tweinmann/siding/internal/fail" +) + +// Container is what siding knows about one container. +type Container struct { + ID string + Name string + Image string + Labels map[string]string + // State is one of created, running, paused, restarting, removing, exited or dead. + State string + // Health is "", starting, healthy or unhealthy. + Health string + // HealthOutput is the output of the last healthcheck. + HealthOutput string + ExitCode int + RestartCount int + OOMKilled bool + Ports []Port +} + +// Port is a published port. +type Port struct { + HostIP string + HostPort int + ContainerPort int +} + +// Network is a Docker network. +type Network struct { + Name string + Labels map[string]string +} + +// Volume is a Docker volume. +type Volume struct { + Name string + Labels map[string]string +} + +// Client talks to the Docker daemon that docker and docker compose use. +type Client struct { + once sync.Once + api *client.Client + err error +} + +// connect resolves the daemon the docker CLI uses: DOCKER_HOST, else the current Docker +// context's endpoint, so siding and docker compose always see the same daemon. +func (c *Client) connect() (*client.Client, error) { + c.once.Do(func() { + opts := []client.Opt{client.FromEnv} + if os.Getenv("DOCKER_HOST") == "" { + if out, err := exec.Command("docker", "context", "inspect", "--format", "{{.Endpoints.docker.Host}}").Output(); err == nil { + if host := strings.TrimSpace(string(out)); host != "" { + opts = append(opts, client.WithHost(host)) + } + } + } + c.api, c.err = client.New(opts...) + if c.err != nil { + c.err = unavailable(c.err) + } + }) + return c.api, c.err +} + +func unavailable(err error) error { + return fail.New(fail.DockerUnavailable, "cannot reach the Docker daemon: %v", err). + WithHint("start Docker, or check DOCKER_HOST and docker context") +} + +// Ping fails with DockerUnavailable when the daemon cannot be reached. +func (c *Client) Ping(ctx context.Context) error { + api, err := c.connect() + if err != nil { + return err + } + if _, err := api.Ping(ctx, client.PingOptions{}); err != nil { + return unavailable(err) + } + return nil +} + +func labelFilters(labels map[string]string) client.Filters { + f := client.Filters{} + for k, v := range labels { + if v == "" { + f.Add("label", k) + } else { + f.Add("label", k+"="+v) + } + } + return f +} + +// Containers returns every container, running or not, that carries all the given labels; an +// empty value matches any value. +func (c *Client) Containers(ctx context.Context, labels map[string]string) ([]Container, error) { + api, err := c.connect() + if err != nil { + return nil, err + } + list, err := api.ContainerList(ctx, client.ContainerListOptions{All: true, Filters: labelFilters(labels)}) + if err != nil { + return nil, unavailable(err) + } + out := make([]Container, 0, len(list.Items)) + for _, s := range list.Items { + ct, err := c.inspect(ctx, api, s.ID) + if err != nil { + if cerrdefs.IsNotFound(err) { + continue // removed meanwhile + } + return nil, err + } + out = append(out, ct) + } + return out, nil +} + +// Container returns the container with this name, or nil when there is none. +func (c *Client) Container(ctx context.Context, name string) (*Container, error) { + api, err := c.connect() + if err != nil { + return nil, err + } + ct, err := c.inspect(ctx, api, name) + if cerrdefs.IsNotFound(err) { + return nil, nil + } + if err != nil { + return nil, err + } + return &ct, nil +} + +func (c *Client) inspect(ctx context.Context, api *client.Client, id string) (Container, error) { + res, err := api.ContainerInspect(ctx, id, client.ContainerInspectOptions{}) + if err != nil { + return Container{}, err + } + return fromInspect(res.Container), nil +} + +func fromInspect(in container.InspectResponse) Container { + ct := Container{ID: in.ID, Name: strings.TrimPrefix(in.Name, "/"), RestartCount: in.RestartCount} + if in.Config != nil { + ct.Image = in.Config.Image + ct.Labels = in.Config.Labels + } + if ct.Labels == nil { + ct.Labels = map[string]string{} + } + if s := in.State; s != nil { + ct.State = string(s.Status) + ct.ExitCode = s.ExitCode + ct.OOMKilled = s.OOMKilled + if h := s.Health; h != nil { + ct.Health = string(h.Status) + if n := len(h.Log); n > 0 && h.Log[n-1] != nil { + ct.HealthOutput = strings.TrimSpace(h.Log[n-1].Output) + } + } + } + if in.NetworkSettings != nil { + for port, bindings := range in.NetworkSettings.Ports { + for _, b := range bindings { + hostPort, err := strconv.Atoi(b.HostPort) + if err != nil { + continue + } + ip := "" + if b.HostIP.IsValid() { + ip = b.HostIP.String() + } + ct.Ports = append(ct.Ports, Port{HostIP: ip, HostPort: hostPort, ContainerPort: int(port.Num())}) + } + } + } + return ct +} + +// Networks returns the networks that carry all the given labels. +func (c *Client) Networks(ctx context.Context, labels map[string]string) ([]Network, error) { + api, err := c.connect() + if err != nil { + return nil, err + } + list, err := api.NetworkList(ctx, client.NetworkListOptions{Filters: labelFilters(labels)}) + if err != nil { + return nil, unavailable(err) + } + out := make([]Network, 0, len(list.Items)) + for _, n := range list.Items { + out = append(out, Network{Name: n.Name, Labels: n.Labels}) + } + return out, nil +} + +// EnsureNetwork creates the network unless it exists and reports whether it created it. +func (c *Client) EnsureNetwork(ctx context.Context, name string, labels map[string]string) (bool, error) { + api, err := c.connect() + if err != nil { + return false, err + } + if _, err := api.NetworkInspect(ctx, name, client.NetworkInspectOptions{}); err == nil { + return false, nil + } else if !cerrdefs.IsNotFound(err) { + return false, unavailable(err) + } + if _, err := api.NetworkCreate(ctx, name, client.NetworkCreateOptions{Driver: "bridge", Labels: labels}); err != nil { + return false, fmt.Errorf("create network %s: %w", name, err) + } + return true, nil +} + +// Volumes returns the volumes that carry all the given labels. +func (c *Client) Volumes(ctx context.Context, labels map[string]string) ([]Volume, error) { + api, err := c.connect() + if err != nil { + return nil, err + } + list, err := api.VolumeList(ctx, client.VolumeListOptions{Filters: labelFilters(labels)}) + if err != nil { + return nil, unavailable(err) + } + out := make([]Volume, 0, len(list.Items)) + for _, v := range list.Items { + out = append(out, Volume{Name: v.Name, Labels: v.Labels}) + } + return out, nil +} + +// Logs returns the last tail lines a container wrote to stdout and stderr, in order. +func (c *Client) Logs(ctx context.Context, id string, tail int) ([]string, error) { + api, err := c.connect() + if err != nil { + return nil, err + } + rc, err := api.ContainerLogs(ctx, id, client.ContainerLogsOptions{ShowStdout: true, ShowStderr: true, Tail: strconv.Itoa(tail)}) + if err != nil { + return nil, fmt.Errorf("logs of %s: %w", id, err) + } + defer func() { _ = rc.Close() }() + // Both streams go into one buffer, so lines keep the order the daemon sent them in. A + // container with a TTY sends a plain stream, not a multiplexed one. + var buf bytes.Buffer + raw, err := readAll(rc) + if err != nil { + return nil, err + } + if _, err := stdcopy.StdCopy(&buf, &buf, bytes.NewReader(raw)); err != nil { + buf.Reset() + buf.Write(raw) + } + var lines []string + sc := bufio.NewScanner(&buf) + sc.Buffer(make([]byte, 64*1024), 1024*1024) + for sc.Scan() { + lines = append(lines, strings.TrimRight(sc.Text(), "\r")) + } + return lines, sc.Err() +} + +func readAll(r interface{ Read([]byte) (int, error) }) ([]byte, error) { + var b bytes.Buffer + _, err := b.ReadFrom(r) + return b.Bytes(), err +} + +// Memory returns the memory a running container uses now, counted as docker stats counts it +// (usage without the inactive page cache). +func (c *Client) Memory(ctx context.Context, id string) (uint64, error) { + api, err := c.connect() + if err != nil { + return 0, err + } + ctx, cancel := context.WithTimeout(ctx, 10*time.Second) + defer cancel() + res, err := api.ContainerStats(ctx, id, client.ContainerStatsOptions{Stream: false}) + if err != nil { + return 0, fmt.Errorf("stats of %s: %w", id, err) + } + defer func() { _ = res.Body.Close() }() + var stats container.StatsResponse + if err := json.NewDecoder(res.Body).Decode(&stats); err != nil { + return 0, fmt.Errorf("stats of %s: %w", id, err) + } + return MemoryUsage(stats.MemoryStats.Usage, stats.MemoryStats.Stats), nil +} + +// MemoryUsage subtracts the inactive page cache from usage: inactive_file on cgroup v2, +// total_inactive_file on cgroup v1. +func MemoryUsage(usage uint64, stats map[string]uint64) uint64 { + for _, key := range []string{"inactive_file", "total_inactive_file"} { + if v, ok := stats[key]; ok && v < usage { + return usage - v + } + } + return usage +} diff --git a/internal/docker/docker_test.go b/internal/docker/docker_test.go new file mode 100644 index 0000000..e9a35f2 --- /dev/null +++ b/internal/docker/docker_test.go @@ -0,0 +1,64 @@ +package docker + +import ( + "net/netip" + "reflect" + "testing" + + "github.com/moby/moby/api/types/container" + "github.com/moby/moby/api/types/network" +) + +func TestFromInspect(t *testing.T) { + t.Parallel() + port, err := network.ParsePort("80/tcp") + if err != nil { + t.Fatal(err) + } + in := container.InspectResponse{ + ID: "abc", + Name: "/siding-proxy", + RestartCount: 2, + Config: &container.Config{Image: "traefik:v3.7.16", Labels: map[string]string{"dev.siding.machine": "m"}}, + State: &container.State{ + Status: container.StateRunning, + ExitCode: 0, + Health: &container.Health{Status: container.Unhealthy, Log: []*container.HealthcheckResult{ + {Output: "old"}, {Output: "connection refused\n"}, + }}, + }, + NetworkSettings: &container.NetworkSettings{Ports: network.PortMap{ + port: {{HostIP: netip.MustParseAddr("127.0.0.1"), HostPort: "8080"}}, + }}, + } + want := Container{ + ID: "abc", Name: "siding-proxy", Image: "traefik:v3.7.16", Labels: map[string]string{"dev.siding.machine": "m"}, + State: "running", Health: "unhealthy", HealthOutput: "connection refused", RestartCount: 2, + Ports: []Port{{HostIP: "127.0.0.1", HostPort: 8080, ContainerPort: 80}}, + } + if got := fromInspect(in); !reflect.DeepEqual(got, want) { + t.Errorf("got %+v\nwant %+v", got, want) + } + if got := fromInspect(container.InspectResponse{}); got.Labels == nil { + t.Error("labels must never be nil") + } +} + +func TestMemoryUsage(t *testing.T) { + t.Parallel() + for _, tt := range []struct { + name string + usage uint64 + stats map[string]uint64 + want uint64 + }{ + {name: "cgroup v2", usage: 1000, stats: map[string]uint64{"inactive_file": 300}, want: 700}, + {name: "cgroup v1", usage: 1000, stats: map[string]uint64{"total_inactive_file": 100}, want: 900}, + {name: "no stats", usage: 1000, want: 1000}, + {name: "cache larger than usage", usage: 10, stats: map[string]uint64{"inactive_file": 20}, want: 10}, + } { + if got := MemoryUsage(tt.usage, tt.stats); got != tt.want { + t.Errorf("%s: %d, want %d", tt.name, got, tt.want) + } + } +} diff --git a/internal/e2e/e2e_test.go b/internal/e2e/e2e_test.go new file mode 100644 index 0000000..3a28e4c --- /dev/null +++ b/internal/e2e/e2e_test.go @@ -0,0 +1,309 @@ +//go:build docker + +// Package e2e runs siding's commands end to end against a real Docker daemon: the +// devcontainer's inner one, through just smoke-stacks. Everything it creates is named +// siding-test-* and removed by exact name; a developer's siding-proxy and network siding are +// never touched. +package e2e + +import ( + "bytes" + "context" + "encoding/json" + "io" + "net/http" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/tweinmann/siding/internal/cli" + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/probe" + "github.com/tweinmann/siding/internal/proxy" + "github.com/tweinmann/siding/internal/stack" +) + +const ( + proxyPort = 18080 + addr = "127.0.0.1:18080" +) + +var testProxy = proxy.Config{ + Project: "siding-test-e2e-proxy", + Container: "siding-test-e2e-proxy", + Network: "siding-test-e2e", + Image: proxy.Image, + DefaultPort: proxyPort, +} + +// A busybox httpd per service answers with the service's name at / and at /api. +const shopCompose = `services: + web: + image: busybox:1.37.0 + command: ["sh", "-c", "mkdir -p /www && echo web > /www/index.html && exec httpd -f -p 80 -h /www"] + ports: ["18081:80"] + labels: {dev.siding.route: "/"} + volumes: [data:/data] + healthcheck: {test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1/"], interval: 1s, retries: 30} + api: + image: busybox:1.37.0 + command: ["sh", "-c", "mkdir -p /www && echo api > /www/api && exec httpd -f -p 80 -h /www"] + expose: ["80"] + labels: {dev.siding.route: "/api"} +volumes: + data: {} +` + +// api crashes after logging a secret from .env; dead is routed but nothing listens on its port. +const brokenCompose = `services: + api: + image: busybox:1.37.0 + command: ["sh", "-c", "echo using token $$API_TOKEN; echo cannot connect to db; exit 1"] + environment: {API_TOKEN: "${API_TOKEN}"} + labels: {dev.siding.route: "/api"} + expose: ["80"] + dead: + image: busybox:1.37.0 + command: ["sleep", "600"] + expose: ["8081"] + labels: {dev.siding.route: "/"} +` + +type machine struct { + t *testing.T + home string +} + +func newMachine(t *testing.T) *machine { + t.Helper() + m := &machine{t: t, home: t.TempDir()} + t.Cleanup(func() { + // By exact name only. + run(t, "", "docker", "compose", "-p", testProxy.Project, "down", "-v", "--remove-orphans") + run(t, "", "docker", "network", "rm", testProxy.Network) + }) + return m +} + +// siding runs one command in dir and returns exit code and stdout; stderr goes to the log. +func (m *machine) siding(dir string, args ...string) (int, string) { + m.t.Helper() + opts := cli.Options{ + Dir: dir, Home: m.home, Getenv: os.Getenv, OSMachineID: stack.OSMachineID, + Git: git.Client{}, Compose: &compose.CLI{}, Docker: &docker.Client{}, + Probe: probe.HTTP{Header: stack.ServiceHeader}, Browser: noBrowser{}, Clock: cli.SystemClock{}, + Proxy: testProxy, + } + var out, errOut bytes.Buffer + code := cli.Execute(context.Background(), cli.New(opts), args, &out, &errOut) + m.t.Logf("siding %s: exit %d\n%s", strings.Join(args, " "), code, errOut.String()) + return code, out.String() +} + +type noBrowser struct{} + +func (noBrowser) Open(string) error { return nil } + +// repo creates a repository with a compose file on branch, one commit ahead of main, and +// removes its stack (project siding-test--) by exact name afterwards. +func repo(t *testing.T, name, composeFile, env, branch string) string { + t.Helper() + dir := filepath.Join(t.TempDir(), name) + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + gitIn(t, dir, "init", "-q", "-b", "main") + gitIn(t, dir, "commit", "-q", "--allow-empty", "-m", "first") + gitIn(t, dir, "switch", "-q", "-c", branch) + for file, content := range map[string]string{"compose.yaml": composeFile, ".env": env} { + if err := os.WriteFile(filepath.Join(dir, file), []byte(content), 0o644); err != nil { + t.Fatal(err) + } + } + gitIn(t, dir, "add", ".") + gitIn(t, dir, "commit", "-q", "-m", "app") + project := "siding-test-" + name + "-" + stack.Slug(branch) + t.Cleanup(func() { run(t, "", "docker", "compose", "-p", project, "down", "-v", "--remove-orphans") }) + return dir +} + +func gitIn(t *testing.T, dir string, args ...string) { + t.Helper() + cmd := exec.Command("git", append([]string{"-C", dir}, args...)...) + cmd.Env = append(os.Environ(), "GIT_CONFIG_GLOBAL=/dev/null", "GIT_CONFIG_NOSYSTEM=1", + "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@example.com", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@example.com") + if out, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("git %v: %v\n%s", args, err, out) + } +} + +// run runs a cleanup command and logs a failure instead of failing: the object may be gone. +func run(t *testing.T, dir string, name string, args ...string) string { + t.Helper() + cmd := exec.Command(name, args...) + cmd.Dir = dir + out, err := cmd.CombinedOutput() + if err != nil { + t.Logf("%s %v: %v\n%s", name, args, err, out) + } + return string(out) +} + +func get(t *testing.T, host, path string) (int, string, string) { + t.Helper() + req, _ := http.NewRequest(http.MethodGet, "http://"+addr+path, nil) + req.Host = host + resp, err := (&http.Client{Timeout: 5 * time.Second}).Do(req) + if err != nil { + t.Fatalf("GET %s%s: %v", host, path, err) + } + defer func() { _ = resp.Body.Close() }() + body, _ := io.ReadAll(resp.Body) + return resp.StatusCode, strings.TrimSpace(string(body)), resp.Header.Get(stack.ServiceHeader) +} + +type status struct { + Kind string `json:"kind"` + Verdict string `json:"verdict"` + Code string `json:"code"` + Failure struct { + Service, Reason, Hint string + Logs []string + } `json:"failure"` +} + +func decode(t *testing.T, stdout string) status { + t.Helper() + var s status + if err := json.Unmarshal([]byte(stdout), &s); err != nil { + t.Fatalf("not a document: %v\n%s", err, stdout) + } + return s +} + +func TestStacks(t *testing.T) { + m := newMachine(t) + + // setup, twice. + if code, out := m.siding(t.TempDir(), "setup", "--json"); code != 0 || !strings.Contains(out, `"state": "created"`) { + t.Fatalf("setup: exit %d\n%s", code, out) + } + if code, out := m.siding(t.TempDir(), "setup", "--json"); code != 0 || !strings.Contains(out, `"state": "unchanged"`) { + t.Fatalf("second setup: exit %d\n%s", code, out) + } + + shop := repo(t, "shop", shopCompose, "", "feature/x") + host := "siding-test-shop.feature-x.localhost" + + t.Run("up --wait is Ready", func(t *testing.T) { + code, out := m.siding(shop, "up", "--repo", "siding-test-shop", "--wait", "--timeout", "90s", "--json") + if s := decode(t, out); code != 0 || s.Verdict != "Ready" { + t.Fatalf("exit %d\n%s", code, out) + } + if got := run(t, shop, "git", "status", "--porcelain"); got != "" { + t.Errorf("the worktree is dirty after up: %q", got) + } + }) + + t.Run("routing by host and path", func(t *testing.T) { + for _, tt := range []struct { + host, path, body, service string + }{ + {host: host, path: "/", body: "web", service: "siding-test-shop-feature-x/web"}, + {host: host, path: "/api", body: "api", service: "siding-test-shop-feature-x/api"}, + {host: "web." + host, path: "/", body: "web", service: "siding-test-shop-feature-x/web"}, + } { + code, body, service := get(t, tt.host, tt.path) + if code != 200 || body != tt.body || service != tt.service { + t.Errorf("%s%s: %d %q, %s %q; want 200 %q from %s", tt.host, tt.path, code, body, stack.ServiceHeader, service, tt.body, tt.service) + } + } + // The app's own 404 carries the header; the proxy's 404 for an unknown host does not. + if code, _, service := get(t, host, "/api/nope"); code != 404 || service == "" { + t.Errorf("the app's 404: %d, header %q", code, service) + } + if code, _, service := get(t, "nope.localhost", "/"); code != 404 || service != "" { + t.Errorf("unknown host: %d, header %q", code, service) + } + }) + + t.Run("no published port", func(t *testing.T) { + out := run(t, "", "docker", "ps", "--filter", "label=com.docker.compose.project=siding-test-shop-feature-x", "--format", "{{.Ports}}") + if strings.Contains(out, "18081") || strings.Contains(out, "->") { + t.Errorf("published ports: %q", out) + } + }) + + t.Run("ps", func(t *testing.T) { + code, out := m.siding(t.TempDir(), "ps", "--json") + if code != 0 || !strings.Contains(out, `"project": "siding-test-shop-feature-x"`) || !strings.Contains(out, `"state": "running"`) { + t.Errorf("exit %d\n%s", code, out) + } + }) + + t.Run("a failure with its log tail", func(t *testing.T) { + broken := repo(t, "broken", brokenCompose, "API_TOKEN=not-a-real-token\n", "main-work") + code, out := m.siding(broken, "up", "--repo", "siding-test-broken", "--wait", "--timeout", "60s", "--json") + s := decode(t, out) + if code != 1 || s.Verdict != "Failed" || s.Failure.Service != "api" || s.Failure.Reason != "Exited" { + t.Fatalf("exit %d\n%s", code, out) + } + logs := strings.Join(s.Failure.Logs, "\n") + if !strings.Contains(logs, "cannot connect to db") || strings.Contains(out, "not-a-real-token") || !strings.Contains(logs, "using token ***") { + t.Errorf("log tail %q", logs) + } + // Nothing listens behind dead's route: the proxy answers 502 on its own. up ended at + // once with Failed, so give Traefik a moment to pick up dead's labels. + var code502 int + var service string + for i := 0; i < 20 && code502 != 502; i++ { + if i > 0 { + time.Sleep(500 * time.Millisecond) + } + code502, _, service = get(t, "siding-test-broken.main-work.localhost", "/") + } + resp := probe.Response{Status: code502, Service: service} + if code502 != 502 || resp.Answered("siding-test-broken-main-work/dead") { + t.Errorf("dead: status %d, header %q; want 502, not answered", code502, service) + } + t.Logf("%s on Traefik's own 502: %q", stack.ServiceHeader, service) + }) + + t.Run("down keeps the data", func(t *testing.T) { + if code, out := m.siding(shop, "down", "--json"); code != 0 || !strings.Contains(out, "siding-test-shop-feature-x-web-1") { + t.Fatalf("exit %d\n%s", code, out) + } + if out := run(t, "", "docker", "volume", "ls", "-q", "--filter", "name=siding-test-shop-feature-x_data"); !strings.Contains(out, "siding-test-shop-feature-x_data") { + t.Errorf("the volume is gone: %q", out) + } + if out := run(t, "", "docker", "ps", "-aq", "--filter", "label=com.docker.compose.project=siding-test-shop-feature-x"); strings.TrimSpace(out) != "" { + t.Errorf("containers left: %q", out) + } + }) + + t.Run("down --volumes keeps unmerged and dirty work", func(t *testing.T) { + if code, out := m.siding(shop, "up", "--repo", "siding-test-shop", "--wait", "--json"); code != 0 { + t.Fatalf("up again: exit %d\n%s", code, out) + } + if code, out := m.siding(shop, "down", "--volumes", "--json"); code != 1 || decode(t, out).Code != "UnmergedBranch" { + t.Errorf("unmerged: exit %d\n%s", code, out) + } + if err := os.WriteFile(filepath.Join(shop, "notes.txt"), []byte("unsaved"), 0o644); err != nil { + t.Fatal(err) + } + if code, out := m.siding(shop, "down", "--volumes", "--json"); code != 1 || decode(t, out).Code != "DirtyWorktree" { + t.Errorf("dirty: exit %d\n%s", code, out) + } + if code, out := m.siding(shop, "down", "--volumes", "--force", "--json"); code != 0 || !strings.Contains(out, "siding-test-shop-feature-x_data") { + t.Fatalf("forced: exit %d\n%s", code, out) + } + if out := run(t, "", "docker", "volume", "ls", "-q", "--filter", "name=siding-test-shop-feature-x_data"); strings.TrimSpace(out) != "" { + t.Errorf("the volume survived --force: %q", out) + } + }) +} diff --git a/internal/fail/fail.go b/internal/fail/fail.go new file mode 100644 index 0000000..f853cf3 --- /dev/null +++ b/internal/fail/fail.go @@ -0,0 +1,60 @@ +// Package fail holds the error type every siding command fails with: a stable code that agents +// may branch on, a message, and an optional hint on what to do next. Codes are part of the +// --json contract and are never renamed within an apiVersion. +package fail + +import "fmt" + +// Error is a failure with a stable code. +type Error struct { + Code string + Message string + Hint string +} + +func (e *Error) Error() string { return e.Message } + +// New returns an Error with a formatted message. +func New(code, format string, args ...any) *Error { + return &Error{Code: code, Message: fmt.Sprintf(format, args...)} +} + +// WithHint returns e with its hint set. +func (e *Error) WithHint(format string, args ...any) *Error { + e.Hint = fmt.Sprintf(format, args...) + return e +} + +// Error codes. The spec cli-contract lists them; add new ones there too. +const ( + // InvalidArguments: unknown command, flag or argument. + InvalidArguments = "InvalidArguments" + // NotAGitRepository: the current directory is not inside a git worktree. + NotAGitRepository = "NotAGitRepository" + // NoBranch: the worktree's HEAD is detached. + NoBranch = "NoBranch" + // ComposeFileNotFound: the worktree has no compose file. + ComposeFileNotFound = "ComposeFileNotFound" + // ComposeFailed: docker compose failed, or is missing or too old. + ComposeFailed = "ComposeFailed" + // DockerUnavailable: the Docker daemon cannot be reached. + DockerUnavailable = "DockerUnavailable" + // ProxyNotRunning: siding-proxy is not running. + ProxyNotRunning = "ProxyNotRunning" + // PortInUse: the proxy's port is held by another process. + PortInUse = "PortInUse" + // InvalidRoute: a dev.siding.route or dev.siding.port label cannot be used. + InvalidRoute = "InvalidRoute" + // AmbiguousPort: a routed service has several container ports and no dev.siding.port. + AmbiguousPort = "AmbiguousPort" + // SlugCollision: another branch or worktree already owns the Compose project. + SlugCollision = "SlugCollision" + // StackNotFound: the worktree has no stack. + StackNotFound = "StackNotFound" + // UnmergedBranch: removing data refused because the branch is not merged. + UnmergedBranch = "UnmergedBranch" + // DirtyWorktree: removing data refused because the worktree has changes. + DirtyWorktree = "DirtyWorktree" + // Internal: an unexpected error without a code of its own. + Internal = "Internal" +) diff --git a/internal/git/git.go b/internal/git/git.go new file mode 100644 index 0000000..a7076fb --- /dev/null +++ b/internal/git/git.go @@ -0,0 +1,147 @@ +// Package git answers what siding needs to know about a worktree by running the git CLI: where +// it is, which branch it is on, and whether removing its stack's data could lose work. +package git + +import ( + "bytes" + "context" + "errors" + "fmt" + "os/exec" + "path/filepath" + "strings" + + "github.com/tweinmann/siding/internal/fail" +) + +// Worktree is a git worktree. +type Worktree struct { + // Top is the absolute path of the worktree's top-level directory. + Top string + // CommonDir is the absolute path of the repository's common git directory, the same for + // every worktree of the repository. + CommonDir string + // Branch is the checked-out branch, without refs/heads/. + Branch string +} + +// RepoName is the name of the directory holding the common git directory: the main worktree's +// directory, the same for every worktree. +func (w Worktree) RepoName() string { + return filepath.Base(filepath.Dir(w.CommonDir)) +} + +// Client runs git. +type Client struct { + // Binary is the git executable; empty means "git" from PATH. + Binary string +} + +func (c Client) run(ctx context.Context, dir string, args ...string) (string, error) { + bin := c.Binary + if bin == "" { + bin = "git" + } + cmd := exec.CommandContext(ctx, bin, append([]string{"-C", dir}, args...)...) + var out, errOut bytes.Buffer + cmd.Stdout = &out + cmd.Stderr = &errOut + err := cmd.Run() + if err != nil { + var exit *exec.ExitError + if errors.As(err, &exit) { + return out.String(), &exitError{code: exit.ExitCode(), stderr: strings.TrimSpace(errOut.String())} + } + return "", fmt.Errorf("run git: %w", err) + } + return out.String(), nil +} + +type exitError struct { + code int + stderr string +} + +func (e *exitError) Error() string { return fmt.Sprintf("git exited %d: %s", e.code, e.stderr) } + +func exitCode(err error) int { + var e *exitError + if errors.As(err, &e) { + return e.code + } + return -1 +} + +// Worktree returns the worktree containing dir. It fails with NotAGitRepository outside a +// worktree and with NoBranch when HEAD is detached. +func (c Client) Worktree(ctx context.Context, dir string) (Worktree, error) { + out, err := c.run(ctx, dir, "rev-parse", "--path-format=absolute", "--show-toplevel", "--git-common-dir") + if err != nil { + if exitCode(err) > 0 { + return Worktree{}, fail.New(fail.NotAGitRepository, "%s is not inside a git worktree", dir). + WithHint("run siding in a git worktree of the repository whose stack you want") + } + return Worktree{}, err + } + lines := strings.Split(strings.TrimSpace(out), "\n") + if len(lines) != 2 { + return Worktree{}, fail.New(fail.NotAGitRepository, "%s is not inside a git worktree", dir) + } + w := Worktree{Top: lines[0], CommonDir: lines[1]} + branch, err := c.run(ctx, dir, "symbolic-ref", "--quiet", "--short", "HEAD") + if err != nil { + if exitCode(err) > 0 { + return Worktree{}, fail.New(fail.NoBranch, "HEAD of %s is detached", w.Top). + WithHint("check out a branch: every stack belongs to one branch") + } + return Worktree{}, err + } + w.Branch = strings.TrimSpace(branch) + return w, nil +} + +// Dirty reports whether git status lists any change in the worktree, untracked files +// included and ignored files excluded. +func (c Client) Dirty(ctx context.Context, top string) (bool, error) { + out, err := c.run(ctx, top, "status", "--porcelain") + if err != nil { + return false, err + } + return strings.TrimSpace(out) != "", nil +} + +// DefaultBranch returns the ref the repository merges into: what origin/HEAD points to, else +// a local main, else a local master. It returns "" when there is none. +func (c Client) DefaultBranch(ctx context.Context, top string) (string, error) { + if out, err := c.run(ctx, top, "symbolic-ref", "--quiet", "refs/remotes/origin/HEAD"); err == nil { + return strings.TrimSpace(out), nil + } else if exitCode(err) < 0 { + return "", err + } + for _, ref := range []string{"refs/heads/main", "refs/heads/master"} { + if _, err := c.run(ctx, top, "rev-parse", "--verify", "--quiet", ref); err == nil { + return ref, nil + } else if exitCode(err) < 0 { + return "", err + } + } + return "", nil +} + +// Merged reports whether the tip of branch is contained in the default branch, and which ref +// that is. Without a default branch it reports false. +func (c Client) Merged(ctx context.Context, top, branch string) (merged bool, base string, err error) { + base, err = c.DefaultBranch(ctx, top) + if err != nil || base == "" { + return false, "", err + } + _, err = c.run(ctx, top, "merge-base", "--is-ancestor", "refs/heads/"+branch, base) + switch code := exitCode(err); { + case err == nil: + return true, base, nil + case code == 1: + return false, base, nil + default: + return false, base, err + } +} diff --git a/internal/git/git_test.go b/internal/git/git_test.go new file mode 100644 index 0000000..d97d4dc --- /dev/null +++ b/internal/git/git_test.go @@ -0,0 +1,158 @@ +package git + +import ( + "context" + "errors" + "os" + "os/exec" + "path/filepath" + "testing" + + "github.com/tweinmann/siding/internal/fail" +) + +// sh runs git in dir with a fixed identity and without the developer's configuration. +func sh(t *testing.T, dir string, args ...string) { + t.Helper() + cmd := exec.Command("git", append([]string{"-C", dir}, args...)...) + cmd.Env = append(os.Environ(), + "GIT_CONFIG_GLOBAL=/dev/null", "GIT_CONFIG_NOSYSTEM=1", + "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@example.com", + "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@example.com") + if out, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("git %v: %v\n%s", args, err, out) + } +} + +// newRepo creates /shop with one commit on main and returns its path. +func newRepo(t *testing.T) string { + t.Helper() + dir := filepath.Join(t.TempDir(), "shop") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + sh(t, dir, "init", "-q", "-b", "main") + sh(t, dir, "commit", "-q", "--allow-empty", "-m", "first") + return dir +} + +func code(err error) string { + var e *fail.Error + if errors.As(err, &e) { + return e.Code + } + return "" +} + +func TestWorktree(t *testing.T) { + t.Parallel() + ctx := context.Background() + main := newRepo(t) + linked := filepath.Join(filepath.Dir(main), "shop-login") + sh(t, main, "worktree", "add", "-q", "-b", "feature/login", linked) + sub := filepath.Join(linked, "web", "src") + if err := os.MkdirAll(sub, 0o755); err != nil { + t.Fatal(err) + } + detached := filepath.Join(filepath.Dir(main), "shop-detached") + sh(t, main, "worktree", "add", "-q", "--detach", detached) + + tests := []struct { + name, dir, top, branch, code string + }{ + {name: "main worktree", dir: main, top: main, branch: "main"}, + {name: "linked worktree from a subdirectory", dir: sub, top: linked, branch: "feature/login"}, + {name: "detached", dir: detached, code: fail.NoBranch}, + {name: "outside", dir: t.TempDir(), code: fail.NotAGitRepository}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + w, err := Client{}.Worktree(ctx, tt.dir) + if tt.code != "" { + if code(err) != tt.code { + t.Fatalf("err %v, want code %s", err, tt.code) + } + return + } + if err != nil { + t.Fatal(err) + } + if w.Top != tt.top || w.Branch != tt.branch || w.RepoName() != "shop" { + t.Errorf("got %+v (repo %s), want top %s, branch %s, repo shop", w, w.RepoName(), tt.top, tt.branch) + } + }) + } +} + +func TestDirty(t *testing.T) { + t.Parallel() + ctx := context.Background() + dir := newRepo(t) + dirty, err := Client{}.Dirty(ctx, dir) + if err != nil || dirty { + t.Fatalf("fresh repository: dirty %v, err %v", dirty, err) + } + // .siding/ ignores itself, so siding's own files never make a worktree dirty. + if err := os.MkdirAll(filepath.Join(dir, ".siding"), 0o755); err != nil { + t.Fatal(err) + } + for name, content := range map[string]string{".siding/.gitignore": "*\n", ".siding/override.yml": "services: {}\n"} { + if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o644); err != nil { + t.Fatal(err) + } + } + if dirty, err := (Client{}).Dirty(ctx, dir); err != nil || dirty { + t.Fatalf("with .siding/: dirty %v, err %v", dirty, err) + } + if err := os.WriteFile(filepath.Join(dir, "new.txt"), []byte("x"), 0o644); err != nil { + t.Fatal(err) + } + if dirty, err := (Client{}).Dirty(ctx, dir); err != nil || !dirty { + t.Fatalf("with an untracked file: dirty %v, err %v", dirty, err) + } +} + +func TestMerged(t *testing.T) { + t.Parallel() + ctx := context.Background() + + t.Run("merged and unmerged against main", func(t *testing.T) { + t.Parallel() + dir := newRepo(t) + sh(t, dir, "branch", "done") + sh(t, dir, "switch", "-q", "-c", "wip") + sh(t, dir, "commit", "-q", "--allow-empty", "-m", "wip") + for branch, want := range map[string]bool{"done": true, "wip": false, "main": true} { + merged, base, err := Client{}.Merged(ctx, dir, branch) + if err != nil || merged != want || base != "refs/heads/main" { + t.Errorf("%s: merged %v, base %q, err %v; want %v against refs/heads/main", branch, merged, base, err, want) + } + } + }) + + t.Run("origin/HEAD wins", func(t *testing.T) { + t.Parallel() + origin := newRepo(t) + clone := filepath.Join(t.TempDir(), "clone") + sh(t, filepath.Dir(clone), "clone", "-q", origin, clone) + merged, base, err := Client{}.Merged(ctx, clone, "main") + if err != nil || !merged || base != "refs/remotes/origin/main" { + t.Errorf("merged %v, base %q, err %v; want true against refs/remotes/origin/main", merged, base, err) + } + }) + + t.Run("no default branch", func(t *testing.T) { + t.Parallel() + dir := filepath.Join(t.TempDir(), "r") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + sh(t, dir, "init", "-q", "-b", "trunk") + sh(t, dir, "commit", "-q", "--allow-empty", "-m", "first") + merged, base, err := Client{}.Merged(ctx, dir, "trunk") + if err != nil || merged || base != "" { + t.Errorf("merged %v, base %q, err %v; want unmerged without a base", merged, base, err) + } + }) +} diff --git a/internal/probe/probe.go b/internal/probe/probe.go new file mode 100644 index 0000000..53e021d --- /dev/null +++ b/internal/probe/probe.go @@ -0,0 +1,53 @@ +// Package probe asks the proxy whether a routed service answers, without DNS: the request goes +// to 127.0.0.1 on the proxy's port with the stack's host in the Host header. +package probe + +import ( + "context" + "io" + "net/http" + "time" +) + +// Response is what the proxy answered. +type Response struct { + Status int + // Service is the value of the X-Siding-Service header, set only on the service's own + // responses. + Service string +} + +// Answered reports whether the response came from service (/) itself and +// not from the proxy: it carries the service header and is not a gateway error. +func (r Response) Answered(service string) bool { + switch r.Status { + case http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout: + return false + } + return r.Service == service +} + +// HTTP probes over plain HTTP. +type HTTP struct { + // Header is the response header naming the service. + Header string +} + +// Get sends GET path with Host: host to addr (host:port) and follows no redirect. +func (p HTTP) Get(ctx context.Context, addr, host, path string) (Response, error) { + ctx, cancel := context.WithTimeout(ctx, 3*time.Second) + defer cancel() + req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://"+addr+path, nil) + if err != nil { + return Response{}, err + } + req.Host = host + c := http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }} + resp, err := c.Do(req) + if err != nil { + return Response{}, err + } + defer func() { _ = resp.Body.Close() }() + _, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 1<<16)) + return Response{Status: resp.StatusCode, Service: resp.Header.Get(p.Header)}, nil +} diff --git a/internal/probe/probe_test.go b/internal/probe/probe_test.go new file mode 100644 index 0000000..4e5ac6a --- /dev/null +++ b/internal/probe/probe_test.go @@ -0,0 +1,55 @@ +package probe + +import ( + "context" + "net/http" + "net/http/httptest" + "strings" + "testing" +) + +func TestGet(t *testing.T) { + t.Parallel() + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.Host != "shop.main.localhost" { + http.Error(w, "404 page not found", http.StatusNotFound) // like Traefik without a router + return + } + switch r.URL.Path { + case "/": + w.Header().Set("X-Siding-Service", "shop-main/web") + http.Redirect(w, r, "/login", http.StatusFound) + case "/api": + w.Header().Set("X-Siding-Service", "shop-main/api") + w.WriteHeader(http.StatusNotFound) // the app's own 404 + default: + w.Header().Set("X-Siding-Service", "shop-main/gone") + w.WriteHeader(http.StatusBadGateway) + } + })) + t.Cleanup(srv.Close) + addr := strings.TrimPrefix(srv.URL, "http://") + p := HTTP{Header: "X-Siding-Service"} + tests := []struct { + name, host, path, service string + answered bool + }{ + {name: "redirect counts", host: "shop.main.localhost", path: "/", service: "shop-main/web", answered: true}, + {name: "the app's 404 counts", host: "shop.main.localhost", path: "/api", service: "shop-main/api", answered: true}, + {name: "the proxy's 404 does not", host: "other.localhost", path: "/", service: "shop-main/web"}, + {name: "a gateway error does not", host: "shop.main.localhost", path: "/x", service: "shop-main/gone"}, + {name: "another service's answer does not", host: "shop.main.localhost", path: "/api", service: "shop-main/web"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + resp, err := p.Get(context.Background(), addr, tt.host, tt.path) + if err != nil { + t.Fatal(err) + } + if got := resp.Answered(tt.service); got != tt.answered { + t.Errorf("response %+v: answered %v, want %v", resp, got, tt.answered) + } + }) + } +} diff --git a/internal/proxy/proxy.go b/internal/proxy/proxy.go new file mode 100644 index 0000000..5c65ccd --- /dev/null +++ b/internal/proxy/proxy.go @@ -0,0 +1,66 @@ +// Package proxy describes siding-proxy: one Traefik with the Docker provider, run as its own +// Compose project, through which every stack is reached by host and path. +package proxy + +import ( + "fmt" + "strings" +) + +// Image is the Traefik image siding-proxy runs; the devcontainer's SIDING_TRAEFIK_IMAGE is the +// same. +const Image = "traefik:v3.7.16" + +// Config names the proxy and its network. Tests use their own, so they never touch a +// developer's siding-proxy. +type Config struct { + Project string // Compose project + Container string // container name + Network string // Docker network every routed service joins + Image string + DefaultPort int // host port on 127.0.0.1 when setup gets no --port +} + +// Default is the proxy of a real installation. +func Default() Config { + return Config{Project: "siding", Container: "siding-proxy", Network: "siding", Image: Image, DefaultPort: 80} +} + +// ComposeFile renders the proxy's compose file. The constraint keeps Traefik to containers +// siding labelled for this proxy's network, so foreign Traefik labels on the same daemon are +// never exposed. +func ComposeFile(c Config, port int, machine string) []byte { + constraint := fmt.Sprintf("LabelRegex(`dev.siding.repo`, `.+`) && Label(`traefik.docker.network`, `%s`)", c.Network) + var b strings.Builder + fmt.Fprintf(&b, `# Written by siding setup. Do not edit: siding setup rewrites it. +services: + traefik: + image: %q + container_name: %q + restart: unless-stopped + command: + - --entrypoints.web.address=:80 + - --providers.docker=true + - --providers.docker.exposedbydefault=false + - --providers.docker.network=%s + - %q + - --ping=true + ports: + - "127.0.0.1:%d:80" + volumes: + - /var/run/docker.sock:/var/run/docker.sock:ro + networks: [proxy] + labels: + dev.siding.machine: %q + healthcheck: + test: ["CMD", "traefik", "healthcheck", "--ping"] + interval: 2s + timeout: 2s + retries: 30 +networks: + proxy: + name: %q + external: true +`, c.Image, c.Container, c.Network, "--providers.docker.constraints="+constraint, port, machine, c.Network) + return []byte(b.String()) +} diff --git a/internal/stack/identity.go b/internal/stack/identity.go new file mode 100644 index 0000000..7e7f9a3 --- /dev/null +++ b/internal/stack/identity.go @@ -0,0 +1,111 @@ +// Package stack derives everything about a stack that follows from its worktree and its +// compose file without asking Docker: names, labels, routes and the override siding writes. +// Everything here is a pure function. +package stack + +import ( + "crypto/sha256" + "encoding/hex" + "strings" +) + +// Labels siding puts on what it creates. The conventions in CLAUDE.md list them. +const ( + LabelRepo = "dev.siding.repo" + LabelBranch = "dev.siding.branch" + LabelWorktree = "dev.siding.worktree" + LabelMachine = "dev.siding.machine" + LabelAgent = "dev.siding.agent" + LabelRoute = "dev.siding.route" + LabelPort = "dev.siding.port" + + // LabelProject is Compose's own label naming a container's project. + LabelProject = "com.docker.compose.project" + // LabelService is Compose's own label naming a container's service. + LabelService = "com.docker.compose.service" +) + +// maxSlug is the longest slug; longer names are cut and get a hash suffix. +const maxSlug = 40 + +// Slug turns a name into lowercase letters, digits and single dashes, at most 40 characters. +func Slug(name string) string { + var b strings.Builder + dash := false + for _, r := range strings.ToLower(name) { + if (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9') { + if dash && b.Len() > 0 { + b.WriteByte('-') + } + dash = false + b.WriteRune(r) + continue + } + dash = true + } + s := b.String() + if s == "" || len(s) > maxSlug { + sum := sha256.Sum256([]byte(name)) + suffix := hex.EncodeToString(sum[:])[:6] + if s == "" { + return "x-" + suffix + } + s = strings.TrimRight(s[:maxSlug-7], "-") + "-" + suffix + } + return s +} + +// Identity names one stack. +type Identity struct { + Repo string // repository name, e.g. shop + Branch string // branch name, e.g. feature/login + Worktree string // absolute path of the worktree + Machine string // stable machine id + Agent string // optional agent name +} + +// RepoSlug is the slug of the repository name. +func (id Identity) RepoSlug() string { return Slug(id.Repo) } + +// BranchSlug is the slug of the branch. +func (id Identity) BranchSlug() string { return Slug(id.Branch) } + +// Project is the Compose project, -. +func (id Identity) Project() string { return id.RepoSlug() + "-" + id.BranchSlug() } + +// Host is the stack's host, ..localhost. +func (id Identity) Host() string { return id.RepoSlug() + "." + id.BranchSlug() + ".localhost" } + +// ServiceHost is the host of one service, ...localhost. +func (id Identity) ServiceHost(service string) string { return Slug(service) + "." + id.Host() } + +// Labels are the labels of the stack's networks and volumes. They never change between two +// ups of one project: Compose recreates a network whose labels changed. +func (id Identity) Labels() map[string]string { + return map[string]string{ + LabelRepo: id.Repo, + LabelBranch: id.Branch, + LabelWorktree: id.Worktree, + LabelMachine: id.Machine, + } +} + +// ContainerLabels are the labels of the stack's containers: Labels plus the agent, if any. +func (id Identity) ContainerLabels() map[string]string { + l := id.Labels() + if id.Agent != "" { + l[LabelAgent] = id.Agent + } + return l +} + +// FromLabels reads an Identity back from a container's labels. +func FromLabels(l map[string]string) Identity { + return Identity{ + Repo: l[LabelRepo], + Branch: l[LabelBranch], + Worktree: l[LabelWorktree], + Machine: l[LabelMachine], + Agent: l[LabelAgent], + } +} diff --git a/internal/stack/identity_test.go b/internal/stack/identity_test.go new file mode 100644 index 0000000..20d6f10 --- /dev/null +++ b/internal/stack/identity_test.go @@ -0,0 +1,126 @@ +package stack + +import ( + "os" + "path/filepath" + "regexp" + "strings" + "testing" +) + +func TestSlug(t *testing.T) { + t.Parallel() + tests := []struct { + name, in, want string + }{ + {name: "plain", in: "main", want: "main"}, + {name: "slash and case", in: "feature/Login_Form", want: "feature-login-form"}, + {name: "runs of symbols", in: "a//b__c..d", want: "a-b-c-d"}, + {name: "leading and trailing", in: "-/x/-", want: "x"}, + {name: "non-ASCII", in: "fix/über", want: "fix-ber"}, + {name: "40 characters stay", in: strings.Repeat("a", 40), want: strings.Repeat("a", 40)}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + if got := Slug(tt.in); got != tt.want { + t.Errorf("Slug(%q) = %q, want %q", tt.in, got, tt.want) + } + }) + } + + hashed := []struct { + name, in string + shape *regexp.Regexp + }{ + {name: "41 characters are cut", in: strings.Repeat("a", 41), shape: regexp.MustCompile(`^a{33}-[0-9a-f]{6}$`)}, + {name: "60 characters are cut", in: strings.Repeat("abcdefghij", 6), shape: regexp.MustCompile(`^(abcdefghij){3}abc-[0-9a-f]{6}$`)}, + {name: "only symbols", in: "///", shape: regexp.MustCompile(`^x-[0-9a-f]{6}$`)}, + } + for _, tt := range hashed { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + if got := Slug(tt.in); !tt.shape.MatchString(got) { + t.Errorf("Slug(%q) = %q, want it to match %s", tt.in, got, tt.shape) + } + }) + } + + t.Run("long names with the same start differ", func(t *testing.T) { + t.Parallel() + long := strings.Repeat("abcdefghij", 6) + if a, b := Slug(long), Slug(long[:59]+"z"); a == b { + t.Errorf("both slugs are %q", a) + } + }) +} + +func TestIdentity(t *testing.T) { + t.Parallel() + id := Identity{Repo: "shop", Branch: "feature/login", Worktree: "/src/shop-login", Machine: "m1"} + if got := id.Project(); got != "shop-feature-login" { + t.Errorf("Project = %q", got) + } + if got := id.Host(); got != "shop.feature-login.localhost" { + t.Errorf("Host = %q", got) + } + if got := id.ServiceHost("Mail_Hog"); got != "mail-hog.shop.feature-login.localhost" { + t.Errorf("ServiceHost = %q", got) + } + if _, ok := id.ContainerLabels()[LabelAgent]; ok { + t.Error("an agent label without an agent") + } + id.Agent = "claude" + if got := id.ContainerLabels()[LabelAgent]; got != "claude" { + t.Errorf("agent label = %q", got) + } + if _, ok := id.Labels()[LabelAgent]; ok { + t.Error("networks and volumes must not carry the agent label") + } + if got := FromLabels(id.ContainerLabels()); got != id { + t.Errorf("FromLabels = %+v, want %+v", got, id) + } + want := map[string]string{LabelRepo: "shop", LabelBranch: "feature/login", LabelWorktree: "/src/shop-login", LabelMachine: "m1"} + for k, v := range want { + if id.Labels()[k] != v { + t.Errorf("label %s = %q, want %q", k, id.Labels()[k], v) + } + } +} + +func TestMachineID(t *testing.T) { + t.Parallel() + + t.Run("from the operating system", func(t *testing.T) { + t.Parallel() + home := t.TempDir() + a, err := MachineID("4c4c4544-not-a-real-id\n", home) + if err != nil { + t.Fatal(err) + } + b, _ := MachineID("4c4c4544-not-a-real-id", home) + if a != b || len(a) != 16 || strings.Contains(a, "4c4c4544") { + t.Errorf("ids %q and %q: want the same 16 hex digits, without the raw id", a, b) + } + if _, err := os.Stat(filepath.Join(home, ".siding")); err == nil { + t.Error("wrote ~/.siding although the operating system has an id") + } + }) + + t.Run("generated once", func(t *testing.T) { + t.Parallel() + home := t.TempDir() + a, err := MachineID("", home) + if err != nil { + t.Fatal(err) + } + b, err := MachineID("", home) + if err != nil || a != b { + t.Fatalf("second call: %q, %v; want %q", b, err, a) + } + saved, _ := os.ReadFile(filepath.Join(home, ".siding", "machine-id")) + if strings.TrimSpace(string(saved)) != a { + t.Errorf("~/.siding/machine-id holds %q, want %q", saved, a) + } + }) +} diff --git a/internal/stack/machineid.go b/internal/stack/machineid.go new file mode 100644 index 0000000..c4ca583 --- /dev/null +++ b/internal/stack/machineid.go @@ -0,0 +1,68 @@ +package stack + +import ( + "crypto/rand" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "io/fs" + "os" + "os/exec" + "path/filepath" + "regexp" + "runtime" + "strings" +) + +// MachineID returns the stable id for dev.siding.machine: a one-way hash of the operating +// system's machine id, so the raw id never appears in a label. Without one it uses, and on +// first use writes, a random id in /.siding/machine-id. +func MachineID(osID, home string) (string, error) { + if osID = strings.TrimSpace(osID); osID != "" { + sum := sha256.Sum256([]byte("siding-machine-id:" + osID)) + return hex.EncodeToString(sum[:])[:16], nil + } + path := filepath.Join(home, ".siding", "machine-id") + if b, err := os.ReadFile(path); err == nil { + if id := strings.TrimSpace(string(b)); id != "" { + return id, nil + } + } else if !errors.Is(err, fs.ErrNotExist) { + return "", fmt.Errorf("read machine id: %w", err) + } + var raw [8]byte + if _, err := rand.Read(raw[:]); err != nil { + return "", err + } + id := hex.EncodeToString(raw[:]) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return "", fmt.Errorf("write machine id: %w", err) + } + if err := os.WriteFile(path, []byte(id+"\n"), 0o644); err != nil { + return "", fmt.Errorf("write machine id: %w", err) + } + return id, nil +} + +var platformUUID = regexp.MustCompile(`"IOPlatformUUID" = "([^"]+)"`) + +// OSMachineID returns the operating system's machine id: /etc/machine-id on Linux, the +// platform UUID on macOS, or "" when there is none. +func OSMachineID() string { + if runtime.GOOS == "darwin" { + out, err := exec.Command("ioreg", "-rd1", "-c", "IOPlatformExpertDevice").Output() + if err != nil { + return "" + } + if m := platformUUID.FindSubmatch(out); m != nil { + return string(m[1]) + } + return "" + } + b, err := os.ReadFile("/etc/machine-id") + if err != nil { + return "" + } + return strings.TrimSpace(string(b)) +} diff --git a/internal/stack/override.go b/internal/stack/override.go new file mode 100644 index 0000000..2c56794 --- /dev/null +++ b/internal/stack/override.go @@ -0,0 +1,117 @@ +package stack + +import ( + "bytes" + "encoding/json" + "fmt" + "maps" + "slices" + "strings" + + "github.com/tweinmann/siding/internal/compose" +) + +// OverrideFile is where siding writes the override, relative to the worktree. +const OverrideFile = ".siding/override.yml" + +// ProxyNetworkKey is the key of the proxy network in the override. It differs from every +// network name a user is likely to pick, so the override never redefines one of theirs. +const ProxyNetworkKey = "siding_proxy" + +// ServiceHeader is the response header every siding router adds, naming /. +// A response that carries it came from the service, not from the proxy (design D9). +const ServiceHeader = "X-Siding-Service" + +// Override renders .siding/override.yml: no published ports, siding's labels on every +// container, network and volume of the project, and for every route the proxy network and +// the Traefik labels. network is the proxy's Docker network. +func Override(p *compose.Project, id Identity, routes []Route, network string) []byte { + var b strings.Builder + project := id.Project() + fmt.Fprintf(&b, "# Written by siding for the Compose project %s. Do not edit: siding up rewrites it.\n", project) + b.WriteString("services:\n") + for _, name := range p.ServiceNames() { + s := p.Services[name] + fmt.Fprintf(&b, " %s:\n", q(name)) + b.WriteString(" ports: !reset []\n") + labels := id.ContainerLabels() + if r, ok := Find(routes, name); ok { + b.WriteString(" networks:\n") + for _, n := range s.Networks { + fmt.Fprintf(&b, " %s: {}\n", q(n)) + } + fmt.Fprintf(&b, " %s: {}\n", ProxyNetworkKey) + maps.Copy(labels, traefikLabels(id, r, network)) + } + writeLabels(&b, " ", labels) + } + + b.WriteString("networks:\n") + for _, key := range sortedKeys(p.Networks) { + if p.Networks[key].External { + continue + } + fmt.Fprintf(&b, " %s:\n", q(key)) + writeLabels(&b, " ", id.Labels()) + } + fmt.Fprintf(&b, " %s:\n name: %s\n external: true\n", ProxyNetworkKey, q(network)) + + volumes := slices.DeleteFunc(sortedKeys(p.Volumes), func(k string) bool { return p.Volumes[k].External }) + if len(volumes) > 0 { + b.WriteString("volumes:\n") + for _, key := range volumes { + fmt.Fprintf(&b, " %s:\n", q(key)) + writeLabels(&b, " ", id.Labels()) + } + } + return []byte(b.String()) +} + +// traefikLabels route the service host to the service and, for a route path, the path on the +// stack's host. Router names are unique per daemon because they start with the project. +func traefikLabels(id Identity, r Route, network string) map[string]string { + name := id.Project() + "-" + Slug(r.Service) + middleware := name + "-id" + l := map[string]string{ + "traefik.enable": "true", + "traefik.docker.network": network, + "traefik.http.services." + name + ".loadbalancer.server.port": fmt.Sprint(r.Port), + "traefik.http.middlewares." + middleware + ".headers.customresponseheaders." + ServiceHeader: id.Project() + "/" + r.Service, + } + router := func(router, rule string) { + prefix := "traefik.http.routers." + router + l[prefix+".rule"] = rule + l[prefix+".entrypoints"] = "web" + l[prefix+".service"] = name + l[prefix+".middlewares"] = middleware + } + router(name, "Host(`"+id.ServiceHost(r.Service)+"`)") + switch r.Path { + case "": + case "/": + router(name+"-route", "Host(`"+id.Host()+"`)") + default: + router(name+"-route", "Host(`"+id.Host()+"`) && PathPrefix(`"+r.Path+"`)") + } + return l +} + +func writeLabels(b *strings.Builder, indent string, labels map[string]string) { + b.WriteString(indent + "labels:\n") + for _, k := range sortedKeys(labels) { + fmt.Fprintf(b, "%s %s: %s\n", indent, q(k), q(labels[k])) + } +} + +// q quotes a YAML scalar. A JSON string is a valid YAML double-quoted scalar. +func q(s string) string { + var b bytes.Buffer + enc := json.NewEncoder(&b) + enc.SetEscapeHTML(false) + _ = enc.Encode(s) + return strings.TrimSuffix(b.String(), "\n") +} + +func sortedKeys[V any](m map[string]V) []string { + return slices.Sorted(maps.Keys(m)) +} diff --git a/internal/stack/redact.go b/internal/stack/redact.go new file mode 100644 index 0000000..c0eda37 --- /dev/null +++ b/internal/stack/redact.go @@ -0,0 +1,90 @@ +package stack + +import ( + "bufio" + "slices" + "strconv" + "strings" +) + +// minSecret is the shortest value siding redacts; shorter ones would wreck every line. +const minSecret = 4 + +// Redactor replaces known secret values in log lines with ***. +type Redactor struct { + values []string // longest first +} + +// NewRedactor redacts the given values except short ones, numbers and booleans. +func NewRedactor(values ...string) *Redactor { + seen := map[string]bool{} + r := &Redactor{} + for _, v := range values { + if len(v) < minSecret || seen[v] || harmless(v) { + continue + } + seen[v] = true + r.values = append(r.values, v) + } + slices.SortFunc(r.values, func(a, b string) int { return len(b) - len(a) }) + return r +} + +func harmless(v string) bool { + switch strings.ToLower(v) { + case "true", "false", "yes", "no", "on", "off": + return true + } + _, err := strconv.ParseFloat(v, 64) + return err == nil +} + +// Line returns line with every known value replaced by ***. +func (r *Redactor) Line(line string) string { + for _, v := range r.values { + line = strings.ReplaceAll(line, v, "***") + } + return line +} + +// Lines redacts every line. +func (r *Redactor) Lines(lines []string) []string { + out := make([]string, len(lines)) + for i, l := range lines { + out[i] = r.Line(l) + } + return out +} + +// DotEnvValues returns the values a .env file defines, read with Compose's rules: KEY=VALUE +// per line, an optional "export ", comments starting with #, single or double quotes. +func DotEnvValues(content string) []string { + var values []string + sc := bufio.NewScanner(strings.NewReader(content)) + for sc.Scan() { + line := strings.TrimSpace(sc.Text()) + if line == "" || strings.HasPrefix(line, "#") { + continue + } + line = strings.TrimPrefix(line, "export ") + _, value, ok := strings.Cut(line, "=") + if !ok { + continue + } + value = strings.TrimSpace(value) + switch { + case len(value) >= 2 && (value[0] == '"' || value[0] == '\''): + if end := strings.IndexByte(value[1:], value[0]); end >= 0 { + value = value[1 : end+1] + } + default: + if i := strings.Index(value, " #"); i >= 0 { + value = strings.TrimSpace(value[:i]) + } + } + if value != "" { + values = append(values, value) + } + } + return values +} diff --git a/internal/stack/redact_test.go b/internal/stack/redact_test.go new file mode 100644 index 0000000..bc277aa --- /dev/null +++ b/internal/stack/redact_test.go @@ -0,0 +1,38 @@ +package stack + +import ( + "reflect" + "testing" +) + +func TestDotEnvValues(t *testing.T) { + t.Parallel() + content := `# a comment +DB_PASSWORD=not-a-real-token +export API_KEY="not-a-real-key" # trailing comment +SINGLE='it''s' +PLAIN=value with spaces # comment +EMPTY= +NO_EQUALS +` + want := []string{"not-a-real-token", "not-a-real-key", "it", "value with spaces"} + if got := DotEnvValues(content); !reflect.DeepEqual(got, want) { + t.Errorf("got %q, want %q", got, want) + } +} + +func TestRedactor(t *testing.T) { + t.Parallel() + r := NewRedactor("not-a-real-token", "not-a-real", "on", "true", "3000", "abc", "development") + tests := []struct{ in, want string }{ + {in: "connecting with not-a-real-token", want: "connecting with ***"}, + {in: "listening on 3000 in development, debug true", want: "listening on 3000 in ***, debug true"}, + {in: "prefix not-a-real only", want: "prefix *** only"}, + {in: "abc stays, it is short", want: "abc stays, it is short"}, + } + for _, tt := range tests { + if got := r.Line(tt.in); got != tt.want { + t.Errorf("Line(%q) = %q, want %q", tt.in, got, tt.want) + } + } +} diff --git a/internal/stack/routing.go b/internal/stack/routing.go new file mode 100644 index 0000000..4d0d9a4 --- /dev/null +++ b/internal/stack/routing.go @@ -0,0 +1,116 @@ +package stack + +import ( + "fmt" + "regexp" + "slices" + "strconv" + "strings" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" +) + +// Route says how the proxy reaches one HTTP service. +type Route struct { + Service string + // Port is the container port the proxy sends requests to. + Port int + // Path is the service's path on the stack's host ("/" for the root), or "" when it has + // only its service host. + Path string +} + +// validPath keeps route paths to characters that are safe inside a Traefik rule. +var validPath = regexp.MustCompile(`^/[A-Za-z0-9._~/-]*$`) + +// Routes derives the routing table of a project: one Route per HTTP service that is not +// hidden, sorted by service. Warnings name services siding cannot route. +func Routes(p *compose.Project) (routes []Route, warnings []string, err error) { + paths := map[string]string{} // path -> service + for _, name := range p.ServiceNames() { + s := p.Services[name] + label, hasLabel := s.Labels[LabelRoute] + label = strings.TrimSpace(label) + if label == "false" { + continue + } + path := "" + if hasLabel && label != "" { + if !validPath.MatchString(label) { + return nil, nil, fail.New(fail.InvalidRoute, "service %s: %s must be \"false\" or a path such as \"/\" or \"/api\", got %q", name, LabelRoute, label) + } + path = label + if path != "/" { + path = strings.TrimRight(path, "/") + } + } + + port, ports, err := httpPort(name, s) + if err != nil { + return nil, nil, err + } + if s.NetworkMode != "" { + if path != "" { + return nil, nil, fail.New(fail.InvalidRoute, "service %s has %s %q but uses network_mode %q, so the proxy cannot reach it", name, LabelRoute, label, s.NetworkMode). + WithHint("remove network_mode or the route label") + } + if port != 0 || len(ports) > 1 { + warnings = append(warnings, fmt.Sprintf("service %s uses network_mode %q and is not routed", name, s.NetworkMode)) + } + continue + } + if path != "" && port == 0 { + if len(ports) > 1 { + return nil, nil, fail.New(fail.AmbiguousPort, "service %s has %s %q and the container ports %s", name, LabelRoute, label, joinInts(ports)). + WithHint("name the HTTP port with the label %s", LabelPort) + } + return nil, nil, fail.New(fail.InvalidRoute, "service %s has %s %q but no container port", name, LabelRoute, label). + WithHint("add expose: [\"\"] or the label %s to the service", LabelPort) + } + if port == 0 { + continue + } + if path != "" { + if other, taken := paths[path]; taken { + return nil, nil, fail.New(fail.InvalidRoute, "services %s and %s both have %s %q", other, name, LabelRoute, path) + } + paths[path] = name + } + routes = append(routes, Route{Service: name, Port: port, Path: path}) + } + return routes, warnings, nil +} + +// httpPort returns the HTTP port of a service, or 0 when it has none or several (returned as +// ports, for the error message). +func httpPort(name string, s compose.Service) (port int, ports []int, err error) { + if v, ok := s.Labels[LabelPort]; ok { + n, err := strconv.Atoi(strings.TrimSpace(v)) + if err != nil || n < 1 || n > 65535 { + return 0, nil, fail.New(fail.InvalidRoute, "service %s: %s must be a port number, got %q", name, LabelPort, v) + } + return n, []int{n}, nil + } + if len(s.Ports) == 1 { + return s.Ports[0], s.Ports, nil + } + return 0, s.Ports, nil +} + +func joinInts(ns []int) string { + s := make([]string, len(ns)) + for i, n := range ns { + s[i] = strconv.Itoa(n) + } + return strings.Join(s, ", ") +} + +// Find returns the route of a service. +func Find(routes []Route, service string) (Route, bool) { + i := slices.IndexFunc(routes, func(r Route) bool { return r.Service == service }) + if i < 0 { + return Route{}, false + } + return routes[i], true +} diff --git a/internal/stack/routing_test.go b/internal/stack/routing_test.go new file mode 100644 index 0000000..21a98b3 --- /dev/null +++ b/internal/stack/routing_test.go @@ -0,0 +1,193 @@ +package stack + +import ( + "errors" + "flag" + "os" + "os/exec" + "path/filepath" + "reflect" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/testutil" +) + +var withCompose = flag.Bool("compose", false, "also check that docker compose accepts every golden override") + +func svc(ports []int, labels map[string]string) compose.Service { + if labels == nil { + labels = map[string]string{} + } + return compose.Service{Ports: ports, Labels: labels, Networks: []string{"default"}} +} + +func TestRoutes(t *testing.T) { + t.Parallel() + tests := []struct { + name string + services map[string]compose.Service + routes []Route + warnings int + code string + }{ + { + name: "root and api", + services: map[string]compose.Service{ + "web": svc([]int{3000}, map[string]string{LabelRoute: "/"}), + "api": svc([]int{8080}, map[string]string{LabelRoute: "/api/"}), + }, + routes: []Route{{Service: "api", Port: 8080, Path: "/api"}, {Service: "web", Port: 3000, Path: "/"}}, + }, + { + name: "one port without label gets a service host", + services: map[string]compose.Service{"mail": svc([]int{8025}, nil)}, + routes: []Route{{Service: "mail", Port: 8025}}, + }, + { + name: "several ports without label are not routed", + services: map[string]compose.Service{"debug": svc([]int{3000, 9229}, nil)}, + }, + { + name: "several ports with the port label", + services: map[string]compose.Service{"web": svc([]int{3000, 9229}, map[string]string{LabelPort: "3000", LabelRoute: "/"})}, + routes: []Route{{Service: "web", Port: 3000, Path: "/"}}, + }, + { + name: "port label without ports", + services: map[string]compose.Service{"web": svc(nil, map[string]string{LabelPort: "80"})}, + routes: []Route{{Service: "web", Port: 80}}, + }, + { + name: "no port", + services: map[string]compose.Service{"worker": svc(nil, nil)}, + }, + { + name: "hidden", + services: map[string]compose.Service{"admin": svc([]int{80}, map[string]string{LabelRoute: "false"})}, + }, + { + name: "two services on one path", + services: map[string]compose.Service{"web": svc([]int{80}, map[string]string{LabelRoute: "/"}), "admin": svc([]int{80}, map[string]string{LabelRoute: "/"})}, + code: fail.InvalidRoute, + }, + { + name: "a route without a port", + services: map[string]compose.Service{"api": svc(nil, map[string]string{LabelRoute: "/api"})}, + code: fail.InvalidRoute, + }, + { + name: "a route with several ports", + services: map[string]compose.Service{"web": svc([]int{3000, 9229}, map[string]string{LabelRoute: "/"})}, + code: fail.AmbiguousPort, + }, + { + name: "not a path", + services: map[string]compose.Service{"web": svc([]int{80}, map[string]string{LabelRoute: "api"})}, + code: fail.InvalidRoute, + }, + { + name: "a path that would break the rule", + services: map[string]compose.Service{"web": svc([]int{80}, map[string]string{LabelRoute: "/a`) || Host(`x"})}, + code: fail.InvalidRoute, + }, + { + name: "a bad port label", + services: map[string]compose.Service{"web": svc([]int{80}, map[string]string{LabelPort: "http"})}, + code: fail.InvalidRoute, + }, + { + name: "network_mode without route warns", + services: map[string]compose.Service{"tools": {Ports: []int{7000}, Labels: map[string]string{}, NetworkMode: "host"}}, + warnings: 1, + }, + { + name: "network_mode with route fails", + services: map[string]compose.Service{"tools": {Ports: []int{7000}, Labels: map[string]string{LabelRoute: "/"}, NetworkMode: "host"}}, + code: fail.InvalidRoute, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + routes, warnings, err := Routes(&compose.Project{Services: tt.services}) + if tt.code != "" { + var e *fail.Error + if !errors.As(err, &e) || e.Code != tt.code { + t.Fatalf("err %v, want code %s", err, tt.code) + } + return + } + if err != nil { + t.Fatal(err) + } + if !reflect.DeepEqual(routes, tt.routes) || len(warnings) != tt.warnings { + t.Errorf("routes %+v, warnings %q; want %+v and %d warnings", routes, warnings, tt.routes, tt.warnings) + } + }) + } +} + +// Each directory in testdata/override holds a captured compose config (config.json, from +// compose.yaml next to it) and the override siding writes for it (override.yml). +func TestOverride(t *testing.T) { + t.Parallel() + id := Identity{Repo: "shop", Branch: "main", Worktree: "/src/shop", Machine: "0123456789abcdef", Agent: "claude"} + for _, dir := range []string{"routing"} { + t.Run(dir, func(t *testing.T) { + t.Parallel() + src := filepath.Join("..", "compose", "testdata", dir) + data, err := os.ReadFile(filepath.Join(src, "config.json")) + if err != nil { + t.Fatal(err) + } + p, err := compose.ParseConfig(data) + if err != nil { + t.Fatal(err) + } + routes, _, err := Routes(p) + if err != nil { + t.Fatal(err) + } + got := Override(p, id, routes, "siding") + golden := filepath.Join("testdata", "override", dir+".yml") + testutil.Golden(t, golden, got) + if *withCompose { + checkWithCompose(t, src, golden) + } + }) + } +} + +// checkWithCompose proves that Compose accepts the override and that the result has no +// published port and keeps the user's networks. +func checkWithCompose(t *testing.T, src, override string) { + t.Helper() + abs, _ := filepath.Abs(override) + cmd := exec.Command("docker", "compose", "-p", "shop-main", "-f", "compose.yaml", "-f", abs, "config", "--format", "json") + cmd.Dir = src + out, err := cmd.Output() + if err != nil { + t.Fatalf("docker compose config with the override: %v", err) + } + merged, err := compose.ParseConfig(out) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(out), `"published"`) { + t.Error("a port is still published") + } + for name, s := range merged.Services { + if s.Labels[LabelRepo] != "shop" { + t.Errorf("%s lacks the repo label", name) + } + } + if got := merged.Services["api"].Networks; !reflect.DeepEqual(got, []string{"backend", "default", ProxyNetworkKey}) { + t.Errorf("api networks %v", got) + } + if got := merged.Services["worker"].Networks; !reflect.DeepEqual(got, []string{"default"}) { + t.Errorf("worker networks %v", got) + } +} diff --git a/internal/stack/testdata/override/routing.yml b/internal/stack/testdata/override/routing.yml new file mode 100644 index 0000000..9c8b29e --- /dev/null +++ b/internal/stack/testdata/override/routing.yml @@ -0,0 +1,134 @@ +# Written by siding for the Compose project shop-main. Do not edit: siding up rewrites it. +services: + "admin": + ports: !reset [] + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "api": + ports: !reset [] + networks: + "backend": {} + "default": {} + siding_proxy: {} + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "traefik.docker.network": "siding" + "traefik.enable": "true" + "traefik.http.middlewares.shop-main-api-id.headers.customresponseheaders.X-Siding-Service": "shop-main/api" + "traefik.http.routers.shop-main-api-route.entrypoints": "web" + "traefik.http.routers.shop-main-api-route.middlewares": "shop-main-api-id" + "traefik.http.routers.shop-main-api-route.rule": "Host(`shop.main.localhost`) && PathPrefix(`/api`)" + "traefik.http.routers.shop-main-api-route.service": "shop-main-api" + "traefik.http.routers.shop-main-api.entrypoints": "web" + "traefik.http.routers.shop-main-api.middlewares": "shop-main-api-id" + "traefik.http.routers.shop-main-api.rule": "Host(`api.shop.main.localhost`)" + "traefik.http.routers.shop-main-api.service": "shop-main-api" + "traefik.http.services.shop-main-api.loadbalancer.server.port": "8080" + "db": + ports: !reset [] + networks: + "backend": {} + siding_proxy: {} + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "traefik.docker.network": "siding" + "traefik.enable": "true" + "traefik.http.middlewares.shop-main-db-id.headers.customresponseheaders.X-Siding-Service": "shop-main/db" + "traefik.http.routers.shop-main-db.entrypoints": "web" + "traefik.http.routers.shop-main-db.middlewares": "shop-main-db-id" + "traefik.http.routers.shop-main-db.rule": "Host(`db.shop.main.localhost`)" + "traefik.http.routers.shop-main-db.service": "shop-main-db" + "traefik.http.services.shop-main-db.loadbalancer.server.port": "5432" + "mail": + ports: !reset [] + networks: + "default": {} + siding_proxy: {} + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "traefik.docker.network": "siding" + "traefik.enable": "true" + "traefik.http.middlewares.shop-main-mail-id.headers.customresponseheaders.X-Siding-Service": "shop-main/mail" + "traefik.http.routers.shop-main-mail.entrypoints": "web" + "traefik.http.routers.shop-main-mail.middlewares": "shop-main-mail-id" + "traefik.http.routers.shop-main-mail.rule": "Host(`mail.shop.main.localhost`)" + "traefik.http.routers.shop-main-mail.service": "shop-main-mail" + "traefik.http.services.shop-main-mail.loadbalancer.server.port": "8025" + "tools": + ports: !reset [] + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "web": + ports: !reset [] + networks: + "default": {} + siding_proxy: {} + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "traefik.docker.network": "siding" + "traefik.enable": "true" + "traefik.http.middlewares.shop-main-web-id.headers.customresponseheaders.X-Siding-Service": "shop-main/web" + "traefik.http.routers.shop-main-web-route.entrypoints": "web" + "traefik.http.routers.shop-main-web-route.middlewares": "shop-main-web-id" + "traefik.http.routers.shop-main-web-route.rule": "Host(`shop.main.localhost`)" + "traefik.http.routers.shop-main-web-route.service": "shop-main-web" + "traefik.http.routers.shop-main-web.entrypoints": "web" + "traefik.http.routers.shop-main-web.middlewares": "shop-main-web-id" + "traefik.http.routers.shop-main-web.rule": "Host(`web.shop.main.localhost`)" + "traefik.http.routers.shop-main-web.service": "shop-main-web" + "traefik.http.services.shop-main-web.loadbalancer.server.port": "3000" + "worker": + ports: !reset [] + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" +networks: + "backend": + labels: + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "default": + labels: + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + siding_proxy: + name: "siding" + external: true +volumes: + "data": + labels: + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" diff --git a/internal/verdict/verdict.go b/internal/verdict/verdict.go new file mode 100644 index 0000000..346fc6a --- /dev/null +++ b/internal/verdict/verdict.go @@ -0,0 +1,191 @@ +// Package verdict decides whether a stack runs: Ready only when every service is up, healthy +// where it has a healthcheck, and answering through the proxy where it has a route path. +// Evaluate is a pure function; status and up --wait call it once per poll. +package verdict + +import ( + "fmt" + "time" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/stack" +) + +// Verdicts. +const ( + Ready = "Ready" + Starting = "Starting" + Failed = "Failed" + TimedOut = "TimedOut" +) + +// Reasons a service is not ready. Part of the --json contract. +const ( + Exited = "Exited" + Unhealthy = "Unhealthy" + Restarting = "Restarting" + Missing = "Missing" + NotAnswering = "NotAnswering" + NotReady = "NotReady" +) + +// restartLimit is how many restarts while waiting count as a crash loop. +const restartLimit = 3 + +// Route is a routed service as the probe sees it. +type Route struct { + URL string + Port int +} + +// Input is everything one evaluation looks at. +type Input struct { + // Services are the compose file's services, in compose order. + Services []string + Containers []docker.Container + // Routes are the services with a route path. + Routes map[string]Route + // Answering holds, per routed service, whether the last probe got its own answer. + Answering map[string]bool + // RestartBase is each container's restart count when waiting began, by container ID. + RestartBase map[string]int + // Final is set when the time is up: what is not ready now is TimedOut. + Final bool + Timeout time.Duration +} + +// Service is the state of one service. +type Service struct { + Name string + State string // container state, or "missing" + Health string + // Answering is set for routed services only. + Answering *bool + Ready bool +} + +// Failure names the service that keeps the stack from being Ready. +type Failure struct { + Service string + Reason string + Hint string +} + +// Result is a verdict with the state of every service. +type Result struct { + Verdict string + Services []Service + Failure *Failure +} + +type outcome int + +const ( + ready outcome = iota + pending + failed +) + +// Evaluate returns the verdict for in. +func Evaluate(in Input) Result { + byService := map[string][]docker.Container{} + for _, c := range in.Containers { + s := c.Labels[stack.LabelService] + byService[s] = append(byService[s], c) + } + var res Result + var firstFailed, firstPending *Failure + allReady := true + for _, name := range in.Services { + svc, out, why := evaluateService(in, name, byService[name]) + res.Services = append(res.Services, svc) + switch out { + case failed: + if firstFailed == nil { + firstFailed = why + } + allReady = false + case pending: + if firstPending == nil { + firstPending = why + } + allReady = false + } + } + switch { + case firstFailed != nil: + res.Verdict, res.Failure = Failed, firstFailed + case allReady: + res.Verdict = Ready + case in.Final: + res.Verdict, res.Failure = TimedOut, firstPending + default: + res.Verdict = Starting + } + return res +} + +// evaluateService looks at every container of a service; the worst one decides. +func evaluateService(in Input, name string, containers []docker.Container) (Service, outcome, *Failure) { + svc := Service{Name: name} + route, routed := in.Routes[name] + if routed { + answering := in.Answering[name] + svc.Answering = &answering + } + if len(containers) == 0 { + svc.State = "missing" + return svc, failed, &Failure{Service: name, Reason: Missing, Hint: "the service has no container; run siding up"} + } + worst, why := ready, (*Failure)(nil) + for _, c := range containers { + svc.State, svc.Health = c.State, c.Health + out, f := evaluateContainer(in, name, c, route, routed) + if out > worst { + worst, why = out, f + } + if worst == failed { + break + } + } + svc.Ready = worst == ready + return svc, worst, why +} + +func evaluateContainer(in Input, name string, c docker.Container, route Route, routed bool) (outcome, *Failure) { + failure := func(reason, format string, args ...any) (outcome, *Failure) { + return failed, &Failure{Service: name, Reason: reason, Hint: fmt.Sprintf(format, args...)} + } + notReady := func() (outcome, *Failure) { + health := "" + if c.Health != "" { + health = ", health " + c.Health + } + return pending, &Failure{Service: name, Reason: NotReady, + Hint: fmt.Sprintf("not ready after %s (state %s%s); raise --timeout, or see its last log lines", in.Timeout, c.State, health)} + } + restarts := c.RestartCount - in.RestartBase[c.ID] + switch { + case c.State == "exited" || c.State == "dead": + if c.ExitCode == 0 && c.State == "exited" { + return ready, nil // a one-off service such as a migration has done its job + } + if c.OOMKilled { + return failure(Exited, "exited with code %d, killed for lack of memory; give it more memory or see its last log lines", c.ExitCode) + } + return failure(Exited, "exited with code %d; see its last log lines", c.ExitCode) + case restarts >= restartLimit || (c.State == "restarting" && in.Final): + return failure(Restarting, "keeps restarting (%d restarts, exit code %d); see its last log lines", c.RestartCount, c.ExitCode) + case c.Health == "unhealthy": + if c.HealthOutput != "" { + return failure(Unhealthy, "its healthcheck fails: %s", c.HealthOutput) + } + return failure(Unhealthy, "its healthcheck fails; see its last log lines") + case c.State != "running" || c.Health == "starting": + return notReady() + case routed && !in.Answering[name]: + return pending, &Failure{Service: name, Reason: NotAnswering, + Hint: fmt.Sprintf("it runs, but %s does not answer through the proxy; check that it listens on port %d on 0.0.0.0, not only on 127.0.0.1", route.URL, route.Port)} + } + return ready, nil +} diff --git a/internal/verdict/verdict_test.go b/internal/verdict/verdict_test.go new file mode 100644 index 0000000..8de31e3 --- /dev/null +++ b/internal/verdict/verdict_test.go @@ -0,0 +1,144 @@ +package verdict + +import ( + "testing" + "time" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/stack" +) + +func ct(service, state, health string, exit int) docker.Container { + return docker.Container{ID: service, Labels: map[string]string{stack.LabelService: service}, State: state, Health: health, ExitCode: exit} +} + +func TestEvaluate(t *testing.T) { + t.Parallel() + routes := map[string]Route{"web": {URL: "http://shop.main.localhost/", Port: 3000}} + tests := []struct { + name string + services []string + containers []docker.Container + answering map[string]bool + base map[string]int + final bool + verdict string + service string + reason string + }{ + { + name: "healthy, answering and done", + services: []string{"db", "migrate", "web"}, + containers: []docker.Container{ct("db", "running", "healthy", 0), ct("migrate", "exited", "", 0), ct("web", "running", "", 0)}, + answering: map[string]bool{"web": true}, + verdict: Ready, + }, + { + name: "running but not answering", + services: []string{"web"}, + containers: []docker.Container{ct("web", "running", "", 0)}, + verdict: Starting, + }, + { + name: "healthcheck starting", + services: []string{"db"}, + containers: []docker.Container{ct("db", "running", "starting", 0)}, + verdict: Starting, + }, + { + name: "exited non-zero", + services: []string{"api", "db"}, + containers: []docker.Container{ct("api", "exited", "", 1), ct("db", "running", "starting", 0)}, + verdict: Failed, service: "api", reason: Exited, + }, + { + name: "unhealthy", + services: []string{"db"}, + containers: []docker.Container{{ID: "db", Labels: map[string]string{stack.LabelService: "db"}, State: "running", Health: "unhealthy", HealthOutput: "pg_isready: no response"}}, + verdict: Failed, service: "db", reason: Unhealthy, + }, + { + name: "missing", + services: []string{"db", "web"}, + containers: []docker.Container{ct("web", "running", "", 0)}, + answering: map[string]bool{"web": true}, + verdict: Failed, service: "db", reason: Missing, + }, + { + name: "restart loop", + services: []string{"api"}, + containers: []docker.Container{{ID: "api", Labels: map[string]string{stack.LabelService: "api"}, State: "running", RestartCount: 5}}, + base: map[string]int{"api": 2}, + verdict: Failed, service: "api", reason: Restarting, + }, + { + name: "restarts from before the wait do not count", + services: []string{"api"}, + containers: []docker.Container{{ID: "api", Labels: map[string]string{stack.LabelService: "api"}, State: "running", RestartCount: 5}}, + base: map[string]int{"api": 5}, + verdict: Ready, + }, + { + name: "timed out not answering", + services: []string{"web"}, + containers: []docker.Container{ct("web", "running", "", 0)}, + final: true, + verdict: TimedOut, service: "web", reason: NotAnswering, + }, + { + name: "timed out starting", + services: []string{"db", "web"}, + containers: []docker.Container{ct("db", "running", "starting", 0), ct("web", "running", "", 0)}, + final: true, + verdict: TimedOut, service: "db", reason: NotReady, + }, + { + name: "the first failure in compose order is reported", + services: []string{"a", "b"}, + containers: []docker.Container{ct("b", "exited", "", 2), ct("a", "exited", "", 3)}, + verdict: Failed, service: "a", reason: Exited, + }, + { + name: "a scaled service: one bad replica fails it", + services: []string{"worker"}, + containers: []docker.Container{ + {ID: "w1", Labels: map[string]string{stack.LabelService: "worker"}, State: "running"}, + {ID: "w2", Labels: map[string]string{stack.LabelService: "worker"}, State: "exited", ExitCode: 137}, + }, + verdict: Failed, service: "worker", reason: Exited, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + res := Evaluate(Input{Services: tt.services, Containers: tt.containers, Routes: routes, + Answering: tt.answering, RestartBase: tt.base, Final: tt.final, Timeout: time.Minute}) + if res.Verdict != tt.verdict { + t.Fatalf("verdict %s, want %s (failure %+v)", res.Verdict, tt.verdict, res.Failure) + } + if tt.reason == "" { + if res.Failure != nil { + t.Errorf("unexpected failure %+v", res.Failure) + } + return + } + if res.Failure == nil || res.Failure.Service != tt.service || res.Failure.Reason != tt.reason || res.Failure.Hint == "" { + t.Errorf("failure %+v, want service %s, reason %s and a hint", res.Failure, tt.service, tt.reason) + } + }) + } +} + +func TestHints(t *testing.T) { + t.Parallel() + res := Evaluate(Input{Services: []string{"db"}, Containers: []docker.Container{ + {ID: "db", Labels: map[string]string{stack.LabelService: "db"}, State: "running", Health: "unhealthy", HealthOutput: "pg_isready: no response"}, + }}) + if want := "its healthcheck fails: pg_isready: no response"; res.Failure.Hint != want { + t.Errorf("hint %q, want %q", res.Failure.Hint, want) + } + res = Evaluate(Input{Services: []string{"api"}, Containers: []docker.Container{ct("api", "exited", "", 1)}}) + if want := "exited with code 1; see its last log lines"; res.Failure.Hint != want { + t.Errorf("hint %q, want %q", res.Failure.Hint, want) + } +} diff --git a/justfile b/justfile index f16cd76..46cdd02 100644 --- a/justfile +++ b/justfile @@ -25,8 +25,9 @@ test: openspec validate --all --strict --no-interactive # Rewrite golden files from the current code after an intended change +# (every package with golden files; a package without the -update flag would fail on it) golden: - go test ./internal/cli -update + go test ./internal/cli ./internal/stack -update # Build a static siding binary for this container (linux) into bin/ build: @@ -35,3 +36,7 @@ build: # Inner daemon: a compose project with a healthcheck, ports reset by an override, Traefik routing smoke-docker: hack/smoke/docker.sh + +# Inner daemon: siding's own commands end to end (setup, up --wait, routing, verdicts, ps, down) +smoke-stacks: + hack/smoke/stacks.sh diff --git a/openspec/changes/phase-1-stacks-and-routing/design.md b/openspec/changes/phase-1-stacks-and-routing/design.md index fda8d42..53dcd8a 100644 --- a/openspec/changes/phase-1-stacks-and-routing/design.md +++ b/openspec/changes/phase-1-stacks-and-routing/design.md @@ -101,6 +101,12 @@ Generated from the normalised config, written as YAML with `!reset` tags: to responses (D9); - top-level: labels on the project's own networks (`default` included) and named volumes, not on external ones; the network `siding` as external. + +Network and volume labels never change between two `up`s of one stack: Compose 5.6 recreates +a network whose labels changed, stopping its containers, and keeps a volume's first labels. +So `dev.siding.agent`, which may differ from one `up` to the next, goes on containers only; +the other labels are fixed for a project (D5 refuses a second branch or worktree). Compose +always runs with stdin closed, so a confirmation prompt fails instead of hanging. Golden files under `internal/stack/testdata/override/` hold input config and expected override. HTTP detection, asked: exactly one container port from `ports` targets and `expose`, or @@ -126,8 +132,9 @@ the service's own when it carries `X-Siding-Service` for that service and its st for an unreachable backend do not count. The probe follows no redirects; a 3xx counts. Alternatives: any status below 500 (cannot tell Traefik's 404 from the app's); asking Traefik's API whether the router exists (needs a second published port). The end-to-end test -checks that Traefik really adds the header to backend responses and not to its own errors; -if it adds it to its 502 too, the status check still rules those out. +shows that Traefik 3.7 adds the header to its own 502 as well (the middleware runs before the +backend fails), so the status rule is what rules those out; its 404 for an unknown host has +no header. ### D10. Verdict A pure function from (services in the config, containers with state, health, exit code, @@ -157,11 +164,14 @@ precise: it guards a stack's data and worktrees, not containers. CLAUDE.md and is the way until `gc` (v0.2) learns more. ### D12. Redaction -Values from the worktree's `.env` and from every `env_file` in the normalised config, 4 -characters or longer, longest first, are replaced by `***` in every log line siding puts into a -document. The `.env` is parsed with Compose's rules for `KEY=VALUE`, quotes and comments; no -values are logged or cached. Shorter values are left, since `***` for `1` or `on` would wreck -every line. Text output of `siding logs` stays raw: it is the user's own terminal, as with +Values from the worktree's `.env` and every service's `environment` in the normalised config, +4 characters or longer, longest first, are replaced by `***` in every log line siding puts +into a document. The normalised config no longer names `env_file`s (Compose 5.6 inlines their +values into `environment`, also with `--no-normalize`), so redacting all of `environment` +covers them and inline values too. Numbers and `true`, `false`, `yes`, `no`, `on`, `off` are +left, as are values shorter than 4 characters, since `***` for them would wreck every line. +The `.env` is parsed with Compose's rules for `KEY=VALUE`, `export`, quotes and comments; no +values are logged or cached. Text output of `siding logs` stays raw: it is the user's own terminal, as with `docker compose logs`. ### D13. `siding skill` (decided here, open for review) @@ -180,6 +190,15 @@ network `siding-test` at port 18080, in temporary git repositories whose `--repo `siding-test-`, and remove everything by exact name in `t.Cleanup`. So the binary has no hidden test switches, and a developer's `siding-proxy` in the devcontainer is never touched. +### D15. No silent port remap in the devcontainer (maintainer's decision) +VS Code forwards a port to another host port when the requested one is taken, without asking: +in the manual test it forwarded 8080 to 8081, and every address siding printed (`:8080`) was +wrong. `"requireLocalPort": true` on 8080 makes VS Code report the conflict instead. Someone +who cannot free 8080 changes the port in `forwardPorts` and `portsAttributes` and runs +`siding setup --port` with the same number. Alternatives: leave the remap and document "look +in the PORTS tab" (every printed address stays wrong); let siding learn the host port (siding +cannot see VS Code's forwarding, and v0.1 has no configuration for an address prefix). + ## Risks / Trade-offs - [Some tools do not resolve `*.localhost`] → browsers and curl do; siding's own probe uses diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/cli-contract/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/cli-contract/spec.md index 5db6ff7..7fc3f5c 100644 --- a/openspec/changes/phase-1-stacks-and-routing/specs/cli-contract/spec.md +++ b/openspec/changes/phase-1-stacks-and-routing/specs/cli-contract/spec.md @@ -47,17 +47,21 @@ with a stable `code` and a human-readable `message`; without `--json` it SHALL p #### Scenario: The error codes - **WHEN** a command fails -- **THEN** its code is one of `InvalidArguments`, `NotAGitRepository`, `NoBranch`, `ComposeFileNotFound`, `ComposeFailed`, `DockerUnavailable`, `ProxyNotRunning`, `PortInUse`, `InvalidRoute`, `AmbiguousPort`, `SlugCollision`, `StackNotFound`, `UnmergedBranch` or `DirtyWorktree` +- **THEN** its code is one of `InvalidArguments`, `NotAGitRepository`, `NoBranch`, `ComposeFileNotFound`, `ComposeFailed`, `DockerUnavailable`, `ProxyNotRunning`, `PortInUse`, `InvalidRoute`, `AmbiguousPort`, `SlugCollision`, `StackNotFound`, `UnmergedBranch`, `DirtyWorktree`, or `Internal` for an unexpected error ### Requirement: Exit codes siding SHALL exit 0 on success and when a stack is Ready, 1 when a command fails or a stack is -Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`), and 3 for -NoCapacity, which v0.1 never returns. +Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`), 3 for +NoCapacity, which v0.1 never returns, and 130 without a message when interrupted. #### Scenario: A verdict decides the exit code - **WHEN** `siding status --wait` ends with the verdict Failed - **THEN** it exits 1, and with Ready it exits 0 +#### Scenario: Interrupted +- **WHEN** the user presses Ctrl-C during `siding logs -f` +- **THEN** siding stops docker compose and exits 130 without an error message + ### Requirement: Version `siding version` SHALL print the version, the commit (when the build knows it), the Go version and the platform; with `--json` as a document of kind `Version` with the fields `version`, diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/dev-environment/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/dev-environment/spec.md index f96a988..d4f6373 100644 --- a/openspec/changes/phase-1-stacks-and-routing/specs/dev-environment/spec.md +++ b/openspec/changes/phase-1-stacks-and-routing/specs/dev-environment/spec.md @@ -19,3 +19,23 @@ and the refusal of `down --volumes`. Its proxy, network and projects MUST carry #### Scenario: Not part of level 1 - **WHEN** `just test` runs without a Docker daemon - **THEN** it passes, because the end-to-end tests only build with the `docker` build tag + +## MODIFIED Requirements + +### Requirement: Devcontainer settings +The devcontainer SHALL be named `siding-devcontainer`, set `SIDING_DEVCONTAINER=1`, forward port +8080 inside to 8080 on the host for the proxy and never to another host port, and set +`git config worktree.useRelativePaths true` when it is created, so worktrees work both inside +the container and on the host. + +#### Scenario: A worktree made in the container +- **WHEN** `git worktree add` runs inside the devcontainer +- **THEN** the worktree's `.git` file and the repository's back-reference use relative paths + +#### Scenario: The host's Git is older than 2.48 +- **WHEN** a worktree with relative paths exists, and Git before 2.48 on the host opens the repository +- **THEN** it refuses with "unknown repository extension found: relativeworktrees", so the host needs Git 2.48 or later + +#### Scenario: Port 8080 is taken on the host +- **WHEN** the devcontainer opens while another process holds port 8080 on the host +- **THEN** VS Code reports that it cannot forward 8080, instead of forwarding to another port that the addresses siding prints do not name diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/stack-identity/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/stack-identity/spec.md index 816eb25..592d35d 100644 --- a/openspec/changes/phase-1-stacks-and-routing/specs/stack-identity/spec.md +++ b/openspec/changes/phase-1-stacks-and-routing/specs/stack-identity/spec.md @@ -76,8 +76,8 @@ with another `dev.siding.worktree` or `dev.siding.branch` label, `up` MUST fail ### Requirement: Labels siding SHALL label every container, network and volume of a stack with `dev.siding.repo`, `dev.siding.branch` (the branch name, not the slug), `dev.siding.worktree` (absolute path) and -`dev.siding.machine`, and with `dev.siding.agent` when `--agent` or `SIDING_AGENT` names one. -External networks and volumes MUST NOT be relabelled. +`dev.siding.machine`, and its containers with `dev.siding.agent` when `--agent` or +`SIDING_AGENT` names one. External networks and volumes MUST NOT be relabelled. #### Scenario: Labels on a running stack - **WHEN** `siding up` has run in `/src/shop-login` on branch `feature/login` diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/stack-status/spec.md b/openspec/changes/phase-1-stacks-and-routing/specs/stack-status/spec.md index ee85bed..e375dbd 100644 --- a/openspec/changes/phase-1-stacks-and-routing/specs/stack-status/spec.md +++ b/openspec/changes/phase-1-stacks-and-routing/specs/stack-status/spec.md @@ -64,8 +64,8 @@ lines of that service. ### Requirement: Waiting `siding status --wait` SHALL check until the verdict is `Ready` or `Failed`, at most for `--timeout` (default 3 minutes). When the time is up it SHALL report `TimedOut`, naming the -first service that is not ready, with reason, hint and log tail; a routed service that never -answered gets the reason `NotAnswering`. +first service that is not ready, with reason, hint and log tail: `NotAnswering` for a routed +service that never answered, `NotReady` for one that is still starting. #### Scenario: Ready in time - **WHEN** `siding status --wait --timeout 60s` runs and the stack becomes ready after 20 seconds diff --git a/openspec/changes/phase-1-stacks-and-routing/tasks.md b/openspec/changes/phase-1-stacks-and-routing/tasks.md index 9119ba5..00ae41e 100644 --- a/openspec/changes/phase-1-stacks-and-routing/tasks.md +++ b/openspec/changes/phase-1-stacks-and-routing/tasks.md @@ -2,67 +2,68 @@ ## 1. Foundations -- [ ] 1.1 Add `Git`, `Compose`, `Docker`, `Probe`, `Browser`, `Clock`, `Home`, `Env` and `Proxy` to `cli.Options` as interfaces or values (D14), with real implementations wired in `cmd/siding/main.go` and fakes in a test helper; verify `just test` passes and no package-level variable is used as a seam -- [ ] 1.2 Add every error code of the spec `cli-contract` with a hint field on `Error`, printed as a second stderr line in text mode and as `hint` in the `Error` document; verify with a table test per code and the updated `error.json` golden -- [ ] 1.3 Extend the exit codes with 2 (TimedOut/Starting) and 3 (NoCapacity, unused) and give `Execute` a way to return a verdict's exit code; verify with a unit test of `Execute` +- [x] 1.1 Add `Git`, `Compose`, `Docker`, `Probe`, `Browser`, `Clock`, `Home`, `Env` and `Proxy` to `cli.Options` as interfaces or values (D14), with real implementations wired in `cmd/siding/main.go` and fakes in a test helper; verify `just test` passes and no package-level variable is used as a seam +- [x] 1.2 Add every error code of the spec `cli-contract` with a hint field on `Error`, printed as a second stderr line in text mode and as `hint` in the `Error` document; verify with a table test per code and the updated `error.json` golden +- [x] 1.3 Extend the exit codes with 2 (TimedOut/Starting) and 3 (NoCapacity, unused) and give `Execute` a way to return a verdict's exit code; verify with a unit test of `Execute` ## 2. Stack identity -- [ ] 2.1 Implement `internal/git`: top level, common dir, current branch (error on detached HEAD), dirty check, merged-into-default-branch check; verify with table tests against temporary repositories (`git init` in `t.TempDir()`), including linked worktrees, detached HEAD, a missing default branch and `.siding/` being ignored -- [ ] 2.2 Implement slugs, project name and hosts in `internal/stack`; verify with a table test covering slashes, case, runs of symbols, leading and trailing symbols, 40/41/60 characters and the hash suffix -- [ ] 2.3 Implement the machine id (D6) with the os id source and `~/.siding/machine-id` from `Options.Home`; verify that two calls agree, that the raw id never appears, and that the fallback file is created once -- [ ] 2.4 Implement the label set (`dev.siding.repo`, `branch`, `worktree`, `machine`, optional `agent` from `--agent` or `SIDING_AGENT`); verify with a unit test +- [x] 2.1 Implement `internal/git`: top level, common dir, current branch (error on detached HEAD), dirty check, merged-into-default-branch check; verify with table tests against temporary repositories (`git init` in `t.TempDir()`), including linked worktrees, detached HEAD, a missing default branch and `.siding/` being ignored +- [x] 2.2 Implement slugs, project name and hosts in `internal/stack`; verify with a table test covering slashes, case, runs of symbols, leading and trailing symbols, 40/41/60 characters and the hash suffix +- [x] 2.3 Implement the machine id (D6) with the os id source and `~/.siding/machine-id` from `Options.Home`; verify that two calls agree, that the raw id never appears, and that the fallback file is created once +- [x] 2.4 Implement the label set (`dev.siding.repo`, `branch`, `worktree`, `machine`, optional `agent` from `--agent` or `SIDING_AGENT`); verify with a unit test ## 3. Compose config, routing and the override -- [ ] 3.1 Implement `internal/compose`: find the compose file (`--file` or Compose's order), run `config --format json`, parse services, ports, `expose`, labels, networks, `network_mode`, healthchecks, named and external volumes and networks, `env_file`; check Compose >= 2.24; verify with golden inputs captured from the devcontainer's Compose in `internal/compose/testdata/` -- [ ] 3.2 Implement HTTP detection and the routing table (service hosts, route paths, `false`, `dev.siding.port`) with the `InvalidRoute` and `AmbiguousPort` errors and the `network_mode` warning; verify with a table test per scenario of the spec `routing` -- [ ] 3.3 Generate `.siding/override.yml` (D7) including `ports: !reset []`, networks, Traefik routers, the `X-Siding-Service` headers middleware and labels on own networks and volumes; verify with golden files in `internal/stack/testdata/override/` (`just golden`) and by checking that `docker compose config` accepts each golden override in the devcontainer -- [ ] 3.4 Add `dev.siding.port` to the label conventions in CLAUDE.md and `openspec/config.yaml`; verify `openspec validate --all --strict` passes +- [x] 3.1 Implement `internal/compose`: find the compose file (`--file` or Compose's order), run `config --format json`, parse services, ports, `expose`, labels, networks, `network_mode`, healthchecks, named and external volumes and networks, `env_file`; check Compose >= 2.24; verify with golden inputs captured from the devcontainer's Compose in `internal/compose/testdata/` +- [x] 3.2 Implement HTTP detection and the routing table (service hosts, route paths, `false`, `dev.siding.port`) with the `InvalidRoute` and `AmbiguousPort` errors and the `network_mode` warning; verify with a table test per scenario of the spec `routing` +- [x] 3.3 Generate `.siding/override.yml` (D7) including `ports: !reset []`, networks, Traefik routers, the `X-Siding-Service` headers middleware and labels on own networks and volumes; verify with golden files in `internal/stack/testdata/override/` (`just golden`) and by checking that `docker compose config` accepts each golden override in the devcontainer +- [x] 3.4 Add `dev.siding.port` to the label conventions in CLAUDE.md and `openspec/config.yaml`; verify `openspec validate --all --strict` passes ## 4. Docker API and the proxy -- [ ] 4.1 Add `github.com/moby/moby/client` and implement `internal/docker`: containers and their state by project label, all siding stacks, published ports of a container, logs tail, one-shot memory stats, network and volume lists by label; `DockerUnavailable` when the daemon is unreachable; verify with unit tests of the mapping from API types to siding's types -- [ ] 4.2 Implement `internal/proxy` and `siding setup [--port] [--json]` (D4): embedded compose file, network `siding`, `127.0.0.1:`, the `dev.siding.repo` constraint, idempotence, recreate on port or image change, `PortInUse`; verify with fake-based tests of the decisions and a `setup.json` golden -- [ ] 4.3 Document `siding setup` in the README (port 80, `--port`, `--port 8080` in the devcontainer, `*.localhost` resolution); verify the commands in the README match `siding setup --help` +- [x] 4.1 Add `github.com/moby/moby/client` and implement `internal/docker`: containers and their state by project label, all siding stacks, published ports of a container, logs tail, one-shot memory stats, network and volume lists by label; `DockerUnavailable` when the daemon is unreachable; verify with unit tests of the mapping from API types to siding's types +- [x] 4.2 Implement `internal/proxy` and `siding setup [--port] [--json]` (D4): embedded compose file, network `siding`, `127.0.0.1:`, the `dev.siding.repo` constraint, idempotence, recreate on port or image change, `PortInUse`; verify with fake-based tests of the decisions and a `setup.json` golden +- [x] 4.3 Document `siding setup` in the README (port 80, `--port`, `--port 8080` in the devcontainer, `*.localhost` resolution); verify the commands in the README match `siding setup --help` ## 5. Up -- [ ] 5.1 Implement `siding up [--file] [--repo] [--agent] [--json] [-- args]`: identity, `ProxyNotRunning`, `SlugCollision`, write `.siding/override.yml` and `.siding/.gitignore`, run Compose with `--remove-orphans` and the passed-through arguments, `ComposeFailed`, Compose progress on stderr; verify with fake-based tests per scenario of the spec `stack-lifecycle` (Up, Override, never edited) and a `stack.json` golden -- [ ] 5.2 Document `siding up` in the README, including that published ports are reset and how to reach a database (`docker compose exec`); verify the example runs as written in the devcontainer +- [x] 5.1 Implement `siding up [--file] [--repo] [--agent] [--json] [-- args]`: identity, `ProxyNotRunning`, `SlugCollision`, write `.siding/override.yml` and `.siding/.gitignore`, run Compose with `--remove-orphans` and the passed-through arguments, `ComposeFailed`, Compose progress on stderr; verify with fake-based tests per scenario of the spec `stack-lifecycle` (Up, Override, never edited) and a `stack.json` golden +- [x] 5.2 Document `siding up` in the README, including that published ports are reset and how to reach a database (`docker compose exec`); verify the example runs as written in the devcontainer ## 6. Verdict and status -- [ ] 6.1 Implement the probe (D9) against `Options.Probe`: `Host` header, no redirects, `X-Siding-Service` and the 502/503/504 rule; verify with an `httptest` server for each case -- [ ] 6.2 Implement `internal/verdict` (D10): Ready, Starting, Failed with `Exited`, `Unhealthy`, `Restarting`, `Missing`, TimedOut with `NotAnswering`, hints, order of evaluation, first failing service in compose order; verify with a table test per scenario of the spec `stack-status` -- [ ] 6.3 Implement `siding status [--wait] [--timeout] [--json]` with polling through `Options.Clock`, the 20-line log tail, `StackNotFound`, exit codes 0/1/2; verify with fake-based tests including a timeout without real waiting, and `status-ready.json`, `status-failed.json` and `status-timedout.json` goldens -- [ ] 6.4 Add `siding up --wait [--timeout]` ending like `status --wait`; verify with a fake-based test that one `Status` document is printed and the exit code follows the verdict +- [x] 6.1 Implement the probe (D9) against `Options.Probe`: `Host` header, no redirects, `X-Siding-Service` and the 502/503/504 rule; verify with an `httptest` server for each case +- [x] 6.2 Implement `internal/verdict` (D10): Ready, Starting, Failed with `Exited`, `Unhealthy`, `Restarting`, `Missing`, TimedOut with `NotAnswering`, hints, order of evaluation, first failing service in compose order; verify with a table test per scenario of the spec `stack-status` +- [x] 6.3 Implement `siding status [--wait] [--timeout] [--json]` with polling through `Options.Clock`, the 20-line log tail, `StackNotFound`, exit codes 0/1/2; verify with fake-based tests including a timeout without real waiting, and `status-ready.json`, `status-failed.json` and `status-timedout.json` goldens +- [x] 6.4 Add `siding up --wait [--timeout]` ending like `status --wait`; verify with a fake-based test that one `Status` document is printed and the exit code follows the verdict ## 7. ps -- [ ] 7.1 Implement `siding ps [--json]`: all stacks by `dev.siding.repo`, state `running`/`partial`/`stopped`, address with the proxy port, memory per stack and in total, `worktreeMissing`; verify with fake-based tests and `stacklist.json` and `stacklist-empty.json` goldens -- [ ] 7.2 Document `siding ps` and `siding status` in the README; verify the shown output matches the text golden of the tests +- [x] 7.1 Implement `siding ps [--json]`: all stacks by `dev.siding.repo`, state `running`/`partial`/`stopped`, address with the proxy port, memory per stack and in total, `worktreeMissing`; verify with fake-based tests and `stacklist.json` and `stacklist-empty.json` goldens +- [x] 7.2 Document `siding ps` and `siding status` in the README; verify the shown output matches the text golden of the tests ## 8. Logs, open and down -- [ ] 8.1 Implement redaction (D12): parse `.env` and `env_file`s, replace values of 4+ characters, longest first; verify with a table test using `not-a-real-token` and quoted, commented and short values -- [ ] 8.2 Implement `siding logs [service…] [-f] [--tail] [--json]` (Compose for text, the Docker API and redaction for the `Logs` document, `InvalidArguments` for `--json -f`); apply redaction to the status log tail too; verify with fake-based tests and a `logs.json` golden -- [ ] 8.3 Implement `siding open [service] [--print] [--json]` with `Options.Browser` (`open` on macOS, `$BROWSER` or `xdg-open` on Linux) and the `Address` document; verify with fake-based tests for port 80, 8080 and an unknown service, and an `address.json` golden -- [ ] 8.4 Implement `siding down [--volumes] [--force] [--dry-run] [--json]` (D11) by project name, the `UnmergedBranch` and `DirtyWorktree` guard for `--volumes`, `StackNotFound`, the `Down` document; verify with fake-based tests per scenario and `down.json` and `down-dryrun.json` goldens -- [ ] 8.5 Make the guardrail "never lose work" precise in CLAUDE.md and `openspec/config.yaml` (data and worktrees are guarded, containers are not) and document `logs`, `open` and `down` in the README; verify `just test` passes +- [x] 8.1 Implement redaction (D12): parse `.env` and `env_file`s, replace values of 4+ characters, longest first; verify with a table test using `not-a-real-token` and quoted, commented and short values +- [x] 8.2 Implement `siding logs [service…] [-f] [--tail] [--json]` (Compose for text, the Docker API and redaction for the `Logs` document, `InvalidArguments` for `--json -f`); apply redaction to the status log tail too; verify with fake-based tests and a `logs.json` golden +- [x] 8.3 Implement `siding open [service] [--print] [--json]` with `Options.Browser` (`open` on macOS, `$BROWSER` or `xdg-open` on Linux) and the `Address` document; verify with fake-based tests for port 80, 8080 and an unknown service, and an `address.json` golden +- [x] 8.4 Implement `siding down [--volumes] [--force] [--dry-run] [--json]` (D11) by project name, the `UnmergedBranch` and `DirtyWorktree` guard for `--volumes`, `StackNotFound`, the `Down` document; verify with fake-based tests per scenario and `down.json` and `down-dryrun.json` goldens +- [x] 8.5 Make the guardrail "never lose work" precise in CLAUDE.md and `openspec/config.yaml` (data and worktrees are guarded, containers are not) and document `logs`, `open` and `down` in the README; verify `just test` passes ## 9. Agent skill -- [ ] 9.1 Write the skill text (`internal/cli/skill/SKILL.md`) and `siding skill [--json]` with the `Skill` document; verify with a golden of the printed text, a test that it names `siding up --wait --json`, `status --json`, `logs --json` and exit codes 0/1/2 and none of `--force`, `--volumes`, `share`, and a `skill.json` golden -- [ ] 9.2 Document installing the skill in the README; verify the shown command produces the file in a temporary directory +- [x] 9.1 Write the skill text (`internal/cli/skill/SKILL.md`) and `siding skill [--json]` with the `Skill` document; verify with a golden of the printed text, a test that it names `siding up --wait --json`, `status --json`, `logs --json` and exit codes 0/1/2 and none of `--force`, `--volumes`, `share`, and a `skill.json` golden +- [x] 9.2 Document installing the skill in the README; verify the shown command produces the file in a temporary directory ## 10. End-to-end and CI -- [ ] 10.1 Write `internal/e2e` with `//go:build docker` (D14): proxy `siding-test-proxy` on network `siding-test` at 18080, temporary repositories with `--repo siding-test-…`, a two-service app with `/` and `/api`, cleanup by exact name in `t.Cleanup`; cases: setup idempotent, `up --wait` Ready, Host and path routing, the service host, `X-Siding-Service` present on app responses and the 502 rule, a failing service giving `Failed` with log tail and a redacted `.env` value, `ps`, `down` keeping volumes, `down --volumes` refused when dirty and unmerged and allowed with `--force`; verify with `just smoke-stacks` in the devcontainer -- [ ] 10.2 Add the recipe `smoke-stacks` (`hack/smoke/stacks.sh` with `require_devcontainer`, running `go test -tags docker ./internal/e2e`) to the justfile and the CI smoke job, and verify afterwards that no `siding-test-*` object is left in the inner daemon and that `just test` still passes without Docker -- [ ] 10.3 Update the dev-environment table in CLAUDE.md and the README with `just smoke-stacks` and `siding setup --port 8080` for the devcontainer; verify the commands run as written -- [ ] 10.4 Run the whole flow by hand in the devcontainer: `just build`, `bin/siding setup --port 8080`, `up --wait` in a sample project, open `http://.localhost:8080` from the host browser, `ps`, `down`; record the result in the phase summary -- [ ] 10.5 Update the status in CLAUDE.md, the README's pre-alpha note and the status in `openspec/config.yaml` to v0.1; verify `just test` passes +- [x] 10.1 Write `internal/e2e` with `//go:build docker` (D14): proxy `siding-test-proxy` on network `siding-test` at 18080, temporary repositories with `--repo siding-test-…`, a two-service app with `/` and `/api`, cleanup by exact name in `t.Cleanup`; cases: setup idempotent, `up --wait` Ready, Host and path routing, the service host, `X-Siding-Service` present on app responses and the 502 rule, a failing service giving `Failed` with log tail and a redacted `.env` value, `ps`, `down` keeping volumes, `down --volumes` refused when dirty and unmerged and allowed with `--force`; verify with `just smoke-stacks` in the devcontainer +- [x] 10.2 Add the recipe `smoke-stacks` (`hack/smoke/stacks.sh` with `require_devcontainer`, running `go test -tags docker ./internal/e2e`) to the justfile and the CI smoke job, and verify afterwards that no `siding-test-*` object is left in the inner daemon and that `just test` still passes without Docker +- [x] 10.3 Update the dev-environment table in CLAUDE.md and the README with `just smoke-stacks` and `siding setup --port 8080` for the devcontainer; verify the commands run as written +- [x] 10.4 Run the whole flow by hand in the devcontainer: `just build`, `bin/siding setup --port 8080`, `up --wait` in a sample project, open `http://.localhost:8080` from the host browser, `ps`, `down`; record the result in the phase summary +- [x] 10.5 Update the status in CLAUDE.md, the README's pre-alpha note and the status in `openspec/config.yaml` to v0.1; verify `just test` passes +- [x] 10.6 Make the devcontainer refuse a silent port remap (D15): `requireLocalPort` for 8080 in `.devcontainer/devcontainer.json`, the requirement `Devcontainer settings` in the `dev-environment` delta, CLAUDE.md and the README; verify `devcontainer.json` parses and `openspec validate --all --strict` passes ## Workflow follow-up diff --git a/openspec/config.yaml b/openspec/config.yaml index 8ad4033..7497aa7 100644 --- a/openspec/config.yaml +++ b/openspec/config.yaml @@ -21,8 +21,8 @@ context: | - Humans see what runs, what it costs and what can go after a merge. - A memory budget keeps the machine usable; later, stacks can run on a remote Docker host. - Status: pre-alpha. Phase 0 (repository, devcontainer, OpenSpec, CI) is done; `siding version` - is the only command. OpenSpec is the only planning source: `openspec/specs/` is what siding + Status: pre-alpha. Phase 0 (repository, devcontainer, OpenSpec, CI) is done; Phase 1 (v0.1: + stacks per worktree, routing through siding-proxy, the verdict of status) is implemented. OpenSpec is the only planning source: `openspec/specs/` is what siding does today, `openspec/changes/` what it is about to do (the roadmap is the `phase--` changes), `openspec/changes/archive/` how it got here. CLAUDE.md holds the working agreements, guardrails, safety rules and conventions in full; read it before @@ -60,8 +60,10 @@ context: | `apiVersion: siding.dev/v1alpha1` and a `kind`; within a version fields are only added; empty fields are omitted, never null; a failure prints `kind: Error` with a stable `code` and a `message`. Exit codes: 0 Ready, 1 Failed, 2 TimedOut, 3 NoCapacity. - 6. Never lose work. Removing a stack or worktree refuses when the branch is unmerged or the - worktree is dirty, unless forced; every destructive command supports `--dry-run`. + 6. Never lose work. Removing a stack's data (its volumes) or a worktree refuses when the + branch is unmerged or the worktree is dirty, unless forced; removing containers and + networks loses nothing and is not guarded (`siding down` keeps volumes, `down --volumes` + is guarded). Every destructive command supports `--dry-run`. 7. Only a human exposes anything to the internet. `siding share` is interactive only and has no `--yes`. 8. Standard building blocks (Docker Compose, Traefik, later Sablier) over custom code. @@ -80,8 +82,11 @@ context: | every stack by Host and path; no stack publishes a port. - Labels on every container, network and volume siding creates or adopts: `dev.siding.repo`, `dev.siding.branch`, `dev.siding.worktree` (absolute path), `dev.siding.machine` (stable - machine id), `dev.siding.agent` (optional). Service labels: `dev.siding.route`, - `dev.siding.seed` (`volume|script|none`). Volume label `dev.siding.shared`. The machine and + machine id); on containers also `dev.siding.agent` (optional; network and volume labels + must not change between two `up`s, since Compose recreates a network whose labels changed). + Service labels: `dev.siding.route` (`/`, `/`, `false`), `dev.siding.port` (the HTTP + port of a service with several), `dev.siding.seed` (`volume|script|none`). Volume label + `dev.siding.shared`. The machine and worktree labels exist from v0.1 so that remote hosts need no migration later. - Development and every test run in a devcontainer with its own Docker daemon (Docker-in-Docker); see the spec `dev-environment`. From db29670e7b9bab952171cb391d54dbd8aeed36af Mon Sep 17 00:00:00 2001 From: tweinmann Date: Wed, 28 Oct 2026 20:46:28 +0000 Subject: [PATCH 3/3] Archive phase 1 and merge its specs The deltas of phase-1-stacks-and-routing become the main specs agent-skill, cli-contract, proxy, routing, stack-identity, stack-lifecycle, stack-list and stack-status, and update dev-environment (Stack smoke test, required local port 8080). Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 10 +- .../.openspec.yaml | 0 .../design.md | 0 .../proposal.md | 0 .../specs/agent-skill/spec.md | 0 .../specs/cli-contract/spec.md | 0 .../specs/dev-environment/spec.md | 0 .../specs/proxy/spec.md | 0 .../specs/routing/spec.md | 0 .../specs/stack-identity/spec.md | 0 .../specs/stack-lifecycle/spec.md | 0 .../specs/stack-list/spec.md | 0 .../specs/stack-status/spec.md | 0 .../tasks.md | 0 openspec/specs/agent-skill/spec.md | 25 ++++ openspec/specs/cli-contract/spec.md | 75 +++++++++++ openspec/specs/dev-environment/spec.md | 27 +++- openspec/specs/proxy/spec.md | 66 ++++++++++ openspec/specs/routing/spec.md | 88 +++++++++++++ openspec/specs/stack-identity/spec.md | 100 ++++++++++++++ openspec/specs/stack-lifecycle/spec.md | 124 ++++++++++++++++++ openspec/specs/stack-list/spec.md | 50 +++++++ openspec/specs/stack-status/spec.md | 84 ++++++++++++ 23 files changed, 642 insertions(+), 7 deletions(-) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/.openspec.yaml (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/design.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/proposal.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/specs/agent-skill/spec.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/specs/cli-contract/spec.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/specs/dev-environment/spec.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/specs/proxy/spec.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/specs/routing/spec.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/specs/stack-identity/spec.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/specs/stack-lifecycle/spec.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/specs/stack-list/spec.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/specs/stack-status/spec.md (100%) rename openspec/changes/{phase-1-stacks-and-routing => archive/2026-10-28-phase-1-stacks-and-routing}/tasks.md (100%) create mode 100644 openspec/specs/agent-skill/spec.md create mode 100644 openspec/specs/cli-contract/spec.md create mode 100644 openspec/specs/proxy/spec.md create mode 100644 openspec/specs/routing/spec.md create mode 100644 openspec/specs/stack-identity/spec.md create mode 100644 openspec/specs/stack-lifecycle/spec.md create mode 100644 openspec/specs/stack-list/spec.md create mode 100644 openspec/specs/stack-status/spec.md diff --git a/CLAUDE.md b/CLAUDE.md index 94de012..557edd3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -14,11 +14,11 @@ changing architecture. ## Status -Phase 0 approved (2026-10-28). Phase 1 (`phase-1-stacks-and-routing`, v0.1) implemented, -waiting for approval (2026-10-28), not yet archived: `setup`, `up [--wait]`, `status [--wait]`, -`ps`, `logs`, `open`, `down [--volumes]`, `skill`, routing by host and path through -`siding-proxy`, `just smoke-stacks`. After approval: `/opsx:archive`, then Phase 2 -(`phase-2-overview-and-budget`, v0.2). +Phase 0 approved (2026-10-28). Phase 1 (v0.1) approved and archived (2026-10-28, +`openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing`): `setup`, `up [--wait]`, +`status [--wait]`, `ps`, `logs`, `open`, `down [--volumes]`, `skill`, routing by host and path +through `siding-proxy`, `just smoke-stacks`. Next: Phase 2 (`phase-2-overview-and-budget`, +v0.2), starting with `/opsx:propose`. ## Working agreements diff --git a/openspec/changes/phase-1-stacks-and-routing/.openspec.yaml b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/.openspec.yaml similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/.openspec.yaml rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/.openspec.yaml diff --git a/openspec/changes/phase-1-stacks-and-routing/design.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/design.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/design.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/design.md diff --git a/openspec/changes/phase-1-stacks-and-routing/proposal.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/proposal.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/proposal.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/proposal.md diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/agent-skill/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/agent-skill/spec.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/specs/agent-skill/spec.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/agent-skill/spec.md diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/cli-contract/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/cli-contract/spec.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/specs/cli-contract/spec.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/cli-contract/spec.md diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/dev-environment/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/dev-environment/spec.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/specs/dev-environment/spec.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/dev-environment/spec.md diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/proxy/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/proxy/spec.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/specs/proxy/spec.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/proxy/spec.md diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/routing/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/routing/spec.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/specs/routing/spec.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/routing/spec.md diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/stack-identity/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-identity/spec.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/specs/stack-identity/spec.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-identity/spec.md diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/stack-list/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-list/spec.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/specs/stack-list/spec.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-list/spec.md diff --git a/openspec/changes/phase-1-stacks-and-routing/specs/stack-status/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-status/spec.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/specs/stack-status/spec.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-status/spec.md diff --git a/openspec/changes/phase-1-stacks-and-routing/tasks.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/tasks.md similarity index 100% rename from openspec/changes/phase-1-stacks-and-routing/tasks.md rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/tasks.md diff --git a/openspec/specs/agent-skill/spec.md b/openspec/specs/agent-skill/spec.md new file mode 100644 index 0000000..2ecde2d --- /dev/null +++ b/openspec/specs/agent-skill/spec.md @@ -0,0 +1,25 @@ +# agent-skill Specification + +## Purpose +Defines the Claude Code skill siding ships, so that a coding agent starts its worktree's stack +and waits for the verdict instead of guessing whether its work runs. + +## Requirements + +### Requirement: siding skill prints the skill +`siding skill` SHALL print a Claude Code skill (`SKILL.md` with name, description and body) on +stdout and write no file. With `--json` it SHALL print a `Skill` document holding the same text. + +#### Scenario: Install the skill +- **WHEN** a user runs `mkdir -p .claude/skills/siding && siding skill > .claude/skills/siding/SKILL.md` +- **THEN** the file is a valid skill named `siding`, and siding wrote nothing itself + +### Requirement: What the skill teaches +The skill SHALL tell the agent to run `siding up --wait --json` after a change that affects the +running app, to read the verdict and exit code, to fix the named service from the failure's +reason, hint and log tail, and to report the address when Ready. It MUST NOT tell the agent to +use `--force`, `--volumes` or `siding share`. + +#### Scenario: The commands in the skill +- **WHEN** the printed skill is read +- **THEN** it names `siding up --wait --json`, `siding status --json`, `siding logs --json` and the exit codes 0, 1 and 2, and none of `--force`, `--volumes` or `share` diff --git a/openspec/specs/cli-contract/spec.md b/openspec/specs/cli-contract/spec.md new file mode 100644 index 0000000..0bdd6ed --- /dev/null +++ b/openspec/specs/cli-contract/spec.md @@ -0,0 +1,75 @@ +# cli-contract Specification + +## Purpose +Defines what agents and scripts can rely on when they call siding: one versioned JSON document +per command with `--json`, stable error codes and fixed exit codes. + +## Requirements + +### Requirement: One JSON document per command +Every siding command SHALL accept `--json` and then print exactly one JSON document on stdout +and nothing else there. The document SHALL carry `apiVersion: siding.dev/v1alpha1` and a +`kind`. Empty fields MUST be left out, never printed as `null`. + +#### Scenario: A command with --json +- **WHEN** `siding version --json` runs +- **THEN** stdout holds one document with `apiVersion` `siding.dev/v1alpha1` and `kind` `Version`, and nothing else + +#### Scenario: Progress of docker compose +- **WHEN** a command with `--json` runs `docker compose`, which reports progress +- **THEN** that progress goes to stderr, and stdout still holds only the document + +#### Scenario: An empty field +- **WHEN** a document has a field without a value, such as a commit unknown to the build +- **THEN** the field is missing from the document + +### Requirement: Compatible within a version +Within `siding.dev/v1alpha1` a document field and an error code MUST NOT be removed, renamed +or given another meaning; new fields, kinds and codes MAY be added. + +#### Scenario: A field is added +- **WHEN** a later release adds a field to the `Status` document +- **THEN** documents still carry `apiVersion` `siding.dev/v1alpha1` and every earlier field with its meaning + +### Requirement: Error document +A command that fails SHALL exit 1. With `--json` it SHALL print a document of kind `Error` +with a stable `code` and a human-readable `message`; without `--json` it SHALL print +`error: ` on stderr, followed by a hint line when there is one. + +#### Scenario: An unknown flag with --json +- **WHEN** `siding version --json --nope` runs +- **THEN** it exits 1 and prints an `Error` document with `code` `InvalidArguments`, even though flag parsing failed before `--json` was read + +#### Scenario: A failure without --json +- **WHEN** `siding up` runs outside a git repository +- **THEN** it exits 1 and prints `error: …` and a hint on stderr, and nothing on stdout + +#### Scenario: The error codes +- **WHEN** a command fails +- **THEN** its code is one of `InvalidArguments`, `NotAGitRepository`, `NoBranch`, `ComposeFileNotFound`, `ComposeFailed`, `DockerUnavailable`, `ProxyNotRunning`, `PortInUse`, `InvalidRoute`, `AmbiguousPort`, `SlugCollision`, `StackNotFound`, `UnmergedBranch`, `DirtyWorktree`, or `Internal` for an unexpected error + +### Requirement: Exit codes +siding SHALL exit 0 on success and when a stack is Ready, 1 when a command fails or a stack is +Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`), 3 for +NoCapacity, which v0.1 never returns, and 130 without a message when interrupted. + +#### Scenario: A verdict decides the exit code +- **WHEN** `siding status --wait` ends with the verdict Failed +- **THEN** it exits 1, and with Ready it exits 0 + +#### Scenario: Interrupted +- **WHEN** the user presses Ctrl-C during `siding logs -f` +- **THEN** siding stops docker compose and exits 130 without an error message + +### Requirement: Version +`siding version` SHALL print the version, the commit (when the build knows it), the Go version +and the platform; with `--json` as a document of kind `Version` with the fields `version`, +`commit`, `goVersion` and `platform`. + +#### Scenario: Text output +- **WHEN** `siding version` runs in a build of commit `0123456789abcdef…` +- **THEN** it prints one line such as `siding v0.1.0 (0123456789ab) go1.27.3 linux/arm64` + +#### Scenario: An argument +- **WHEN** `siding version extra` runs +- **THEN** it fails with `InvalidArguments` diff --git a/openspec/specs/dev-environment/spec.md b/openspec/specs/dev-environment/spec.md index aada57d..5eda724 100644 --- a/openspec/specs/dev-environment/spec.md +++ b/openspec/specs/dev-environment/spec.md @@ -32,8 +32,9 @@ stack siding's tests create, and MUST NOT mount or reach the host's Docker socke ### Requirement: Devcontainer settings The devcontainer SHALL be named `siding-devcontainer`, set `SIDING_DEVCONTAINER=1`, forward port -8080 inside to 8080 on the host for the proxy, and set `git config worktree.useRelativePaths true` -when it is created, so worktrees work both inside the container and on the host. +8080 inside to 8080 on the host for the proxy and never to another host port, and set +`git config worktree.useRelativePaths true` when it is created, so worktrees work both inside +the container and on the host. #### Scenario: A worktree made in the container - **WHEN** `git worktree add` runs inside the devcontainer @@ -43,6 +44,10 @@ when it is created, so worktrees work both inside the container and on the host. - **WHEN** a worktree with relative paths exists, and Git before 2.48 on the host opens the repository - **THEN** it refuses with "unknown repository extension found: relativeworktrees", so the host needs Git 2.48 or later +#### Scenario: Port 8080 is taken on the host +- **WHEN** the devcontainer opens while another process holds port 8080 on the host +- **THEN** VS Code reports that it cannot forward 8080, instead of forwarding to another port that the addresses siding prints do not name + ### Requirement: Guard in the hack scripts Every script in `hack/` except `hack/nuke.sh` MUST abort unless `SIDING_DEVCONTAINER=1` and the Docker daemon reports the container's own host name. @@ -86,6 +91,24 @@ port the compose file publishes, and that Traefik with the Docker provider on th - **WHEN** the demo project runs with the override - **THEN** none of its containers has a published port, although the compose file publishes one +### Requirement: Stack smoke test +`just smoke-stacks` SHALL run siding's own commands end to end against the inner daemon: +`setup`, `up --wait`, routing by host and path, a `Failed` verdict with log tail, `ps`, `down` +and the refusal of `down --volumes`. Its proxy, network and projects MUST carry the prefix +`siding-test-`, and its proxy MUST listen on 18080. + +#### Scenario: Stack smoke test in CI +- **WHEN** a pull request is opened or pushed to, or a commit is pushed to `main` +- **THEN** CI runs `just smoke-stacks` inside the devcontainer, against its own daemon + +#### Scenario: It leaves nothing behind +- **WHEN** `just smoke-stacks` ends, passing or failing +- **THEN** no container, network or volume whose name starts with `siding-test-` is left in the inner daemon, and `siding-proxy` and the network `siding` are untouched + +#### Scenario: Not part of level 1 +- **WHEN** `just test` runs without a Docker daemon +- **THEN** it passes, because the end-to-end tests only build with the `docker` build tag + ### Requirement: Test stacks Tests SHALL create stacks only as Compose projects whose name starts with `siding-test-`, and MUST remove them by their exact project name, also when the test fails. The smoke test's proxy diff --git a/openspec/specs/proxy/spec.md b/openspec/specs/proxy/spec.md new file mode 100644 index 0000000..8a142cc --- /dev/null +++ b/openspec/specs/proxy/spec.md @@ -0,0 +1,66 @@ +# proxy Specification + +## Purpose +Defines the one shared proxy through which every stack is reached: `siding-proxy`, a Traefik +on the network `siding`, started by `siding setup` and bound to the loopback interface only. + +## Requirements + +### Requirement: Setup starts the proxy +`siding setup` SHALL create the network `siding` if it is missing and run the Compose project +`siding` with one container `siding-proxy`: Traefik with the Docker provider on the network +`siding`, restarting unless stopped. It SHALL print the proxy's address, with `--json` as a +`Setup` document. + +#### Scenario: First setup +- **WHEN** `siding setup` runs on a machine without the proxy +- **THEN** the network `siding` and the running container `siding-proxy` exist, and it prints `http://*.localhost` as the address + +#### Scenario: JSON +- **WHEN** `siding setup --json` runs +- **THEN** it prints a `Setup` document with the proxy's container name, network, image and port + +### Requirement: Loopback port +The proxy SHALL publish exactly one port, on `127.0.0.1` only, `80` unless `--port ` names +another. siding MUST read the proxy's port from the running container, not from a file. + +#### Scenario: Default port +- **WHEN** `siding setup` has run +- **THEN** `siding-proxy` publishes `127.0.0.1:80` and no other port, and nothing listens for it on other interfaces + +#### Scenario: Another port +- **WHEN** `siding setup --port 8080` has run and `siding open` runs later +- **THEN** the address it opens is `http://:8080` + +#### Scenario: The port is taken +- **WHEN** another process holds `127.0.0.1:80` and `siding setup` runs +- **THEN** it fails with `PortInUse` and a hint to use `--port` + +### Requirement: Setup is idempotent +`siding setup` SHALL change nothing when the proxy already runs as requested, and SHALL +recreate only the proxy container when its port or image differs. Running stacks MUST keep +running and be routed again once the proxy is back. + +#### Scenario: Second run +- **WHEN** `siding setup` runs twice +- **THEN** the second run reports the proxy as already running and recreates nothing + +#### Scenario: Port change with running stacks +- **WHEN** a stack runs and `siding setup --port 8080` follows `siding setup` +- **THEN** the stack's containers are not restarted and it answers on port 8080 + +### Requirement: Only siding stacks are routed +The proxy MUST route only containers that carry `dev.siding.repo` and enable Traefik, so that +other containers with Traefik labels on the same daemon are never exposed through it. + +#### Scenario: A foreign Traefik label +- **WHEN** a container without siding labels carries `traefik.enable=true` and a Host rule +- **THEN** the proxy answers 404 for that host + +### Requirement: Stacks need the proxy +`up` and `status` SHALL fail with `ProxyNotRunning` and the hint `siding setup` when +`siding-proxy` is not running. + +#### Scenario: up before setup +- **WHEN** `siding up` runs on a machine without the proxy +- **THEN** it fails with `ProxyNotRunning` before starting any container diff --git a/openspec/specs/routing/spec.md b/openspec/specs/routing/spec.md new file mode 100644 index 0000000..1dc1921 --- /dev/null +++ b/openspec/specs/routing/spec.md @@ -0,0 +1,88 @@ +# routing Specification + +## Purpose +Defines how the services of a stack are reached through the proxy: which services count as +HTTP services, how the label `dev.siding.route` maps them to the stack's host and paths, and +how `siding open` finds the address. + +## Requirements + +### Requirement: HTTP services +A service SHALL be an HTTP service when the user's compose file gives it exactly one distinct +container port, from the targets of `ports` and from `expose`, or when its label +`dev.siding.port` names one. A service with several ports and no `dev.siding.port` is not one. + +#### Scenario: One published port +- **WHEN** a service publishes `"3000:3000"` +- **THEN** it is an HTTP service on container port 3000, and the port is no longer published + +#### Scenario: Several ports with the label +- **WHEN** a service exposes 3000 and 9229 and carries `dev.siding.port: "3000"` +- **THEN** it is an HTTP service on container port 3000 + +#### Scenario: No port +- **WHEN** a worker service has neither `ports` nor `expose` nor `dev.siding.port` +- **THEN** it is not routed + +### Requirement: Service host +Every HTTP service SHALL answer at `...localhost`, unless it +carries `dev.siding.route: "false"`. + +#### Scenario: An unlabelled HTTP service +- **WHEN** the service `mail` exposes 8025 and has no route label, in the stack `shop` on branch `main` +- **THEN** `mail.shop.main.localhost` reaches it on 8025 + +#### Scenario: Hidden +- **WHEN** a service carries `dev.siding.route: "false"` +- **THEN** no host or path reaches it through the proxy + +### Requirement: Route label +`dev.siding.route: "/"` SHALL make an HTTP service the root of the stack's host, and +`dev.siding.route: "/"` SHALL mount it on the stack's host under that path prefix, +passing the path on unchanged. The longest matching prefix MUST win. + +#### Scenario: Root and API +- **WHEN** `web` carries `dev.siding.route: "/"` and `api` carries `dev.siding.route: "/api"` in the stack `shop` on `main` +- **THEN** `shop.main.localhost/` reaches `web`, and `shop.main.localhost/api/users` reaches `api` with the path `/api/users` + +#### Scenario: No root +- **WHEN** no service carries `dev.siding.route: "/"` +- **THEN** a path on the stack's host that no route covers answers 404 + +### Requirement: Invalid routes +`siding up` SHALL fail with `InvalidRoute`, naming the service, and start nothing when a route +label is neither `false` nor an absolute path, when two services claim the same path, or when +a service with a route is not an HTTP service. + +#### Scenario: Two services on one path +- **WHEN** `web` and `admin` both carry `dev.siding.route: "/"` +- **THEN** `siding up` fails with `InvalidRoute` naming both + +#### Scenario: A route without a port +- **WHEN** a service carries `dev.siding.route: "/api"` but has no container port +- **THEN** `siding up` fails with `InvalidRoute` and a hint to add `expose` or `dev.siding.port` + +#### Scenario: A route with several ports +- **WHEN** a service carries `dev.siding.route: "/"` and exposes 3000 and 9229 without `dev.siding.port` +- **THEN** `siding up` fails with `AmbiguousPort` naming the ports + +### Requirement: Services that cannot join the proxy network +A service with a `network_mode` SHALL NOT be routed; `up` SHALL warn about it on stderr, and +fail with `InvalidRoute` when it carries a route path. + +#### Scenario: network_mode host +- **WHEN** a service uses `network_mode: host` and has no route label +- **THEN** `siding up` starts it unrouted and prints a warning + +### Requirement: Open +`siding open [service]` SHALL open the stack's address, or the service's host, in the default +browser; with `--print` or `--json` it SHALL only print it, as text or as an `Address` +document. The address MUST include the proxy's port unless it is 80. + +#### Scenario: Print the address +- **WHEN** the proxy listens on 8080 and `siding open --print` runs in the stack `shop` on `main` +- **THEN** it prints `http://shop.main.localhost:8080/` + +#### Scenario: Unknown service +- **WHEN** `siding open nope` runs and the stack has no HTTP service `nope` +- **THEN** it fails with `InvalidArguments` and lists the HTTP services diff --git a/openspec/specs/stack-identity/spec.md b/openspec/specs/stack-identity/spec.md new file mode 100644 index 0000000..d831c60 --- /dev/null +++ b/openspec/specs/stack-identity/spec.md @@ -0,0 +1,100 @@ +# stack-identity Specification + +## Purpose +Defines how siding derives a stack's identity from a git worktree: repository name, branch +slug, Compose project, host names, machine id and the labels that let Docker answer every +later question about the stack. + +## Requirements + +### Requirement: One stack per worktree +siding SHALL treat the git worktree containing the current directory as the stack's worktree, +identified by its absolute top-level path. Every command that acts on a stack MUST act on that +worktree's stack. + +#### Scenario: Run from a subdirectory +- **WHEN** `siding status` runs in `/src/shop-login/web/src`, inside the worktree `/src/shop-login` +- **THEN** it reports the stack of `/src/shop-login` + +#### Scenario: Outside a repository +- **WHEN** a stack command runs outside any git worktree +- **THEN** it fails with `NotAGitRepository` + +### Requirement: Repository name +The repository name SHALL be the name of the directory that holds the repository's common git +directory (the main worktree), the same for every worktree of the repository. `--repo ` +MUST override it. + +#### Scenario: Two worktrees of one repository +- **WHEN** `/src/shop` is the main worktree and `/src/shop-login` a linked worktree of it +- **THEN** both have the repository name `shop` + +#### Scenario: Override +- **WHEN** `siding up --repo store` runs in `/src/shop` +- **THEN** the repository name is `store` + +### Requirement: Slugs +The repository name and the branch SHALL each be turned into a slug: lowercase, every run of +characters other than `a-z` and `0-9` replaced by one `-`, no leading or trailing `-`. A slug +longer than 40 characters MUST be cut to 33 characters, followed by `-` and the first 6 hex +digits of the SHA-256 of the original name. + +#### Scenario: A branch with a slash +- **WHEN** the branch is `feature/Login_Form` +- **THEN** its slug is `feature-login-form` + +#### Scenario: A long branch +- **WHEN** the branch name is 60 characters long +- **THEN** its slug is 40 characters long and ends in `-` and 6 hex digits that differ for another branch with the same first 33 characters + +### Requirement: Branch required +The worktree SHALL be on a branch. In a detached HEAD a stack command MUST fail with `NoBranch` +and a hint to check out a branch. + +#### Scenario: Detached HEAD +- **WHEN** `siding up` runs in a worktree whose HEAD is detached +- **THEN** it fails with `NoBranch` and starts nothing + +### Requirement: Project name and host +The Compose project SHALL be named `-` and the stack's host SHALL be +`..localhost`. + +#### Scenario: Names +- **WHEN** the repository is `shop` and the branch `feature/login` +- **THEN** the project is `shop-feature-login` and the host `shop.feature-login.localhost` + +### Requirement: Slug collision +siding MUST NOT let two worktrees or branches share one project. If the project already exists +with another `dev.siding.worktree` or `dev.siding.branch` label, `up` MUST fail with +`SlugCollision`, naming the other worktree and branch, and change nothing. + +#### Scenario: feature/a and feature-a +- **WHEN** a stack for branch `feature/a` runs and `siding up` runs in a worktree on branch `feature-a` +- **THEN** it fails with `SlugCollision` and the running stack is untouched + +### Requirement: Labels +siding SHALL label every container, network and volume of a stack with `dev.siding.repo`, +`dev.siding.branch` (the branch name, not the slug), `dev.siding.worktree` (absolute path) and +`dev.siding.machine`, and its containers with `dev.siding.agent` when `--agent` or +`SIDING_AGENT` names one. External networks and volumes MUST NOT be relabelled. + +#### Scenario: Labels on a running stack +- **WHEN** `siding up` has run in `/src/shop-login` on branch `feature/login` +- **THEN** every container, the project's networks and its named volumes carry `dev.siding.repo=shop`, `dev.siding.branch=feature/login` and `dev.siding.worktree=/src/shop-login` + +#### Scenario: An agent name +- **WHEN** `SIDING_AGENT=claude siding up` runs +- **THEN** the containers also carry `dev.siding.agent=claude` + +### Requirement: Machine id +`dev.siding.machine` SHALL be a stable id of the machine siding runs on: derived from the +operating system's machine id by a one-way hash, never the raw id. Without an operating system +id, siding MUST generate one and keep it in `~/.siding/machine-id`. + +#### Scenario: Two runs +- **WHEN** `siding up` runs twice on the same machine +- **THEN** both stacks carry the same `dev.siding.machine` + +#### Scenario: No operating system id +- **WHEN** neither `/etc/machine-id` nor the macOS platform UUID is available +- **THEN** siding writes a random id to `~/.siding/machine-id` and uses it from then on diff --git a/openspec/specs/stack-lifecycle/spec.md b/openspec/specs/stack-lifecycle/spec.md new file mode 100644 index 0000000..3de30f3 --- /dev/null +++ b/openspec/specs/stack-lifecycle/spec.md @@ -0,0 +1,124 @@ +# stack-lifecycle Specification + +## Purpose +Defines how a worktree's stack is started, inspected through its logs and taken down: the +user's compose file stays untouched, siding adds one override, and nothing removes unmerged or +uncommitted work without being forced. + +## Requirements + +### Requirement: Compose file +siding SHALL use the compose file Compose itself would pick in the worktree's top-level +directory (`compose.yaml`, `compose.yml`, `docker-compose.yaml`, `docker-compose.yml`), or the +one named by `--file`. Without one, stack commands MUST fail with `ComposeFileNotFound`. + +#### Scenario: The default file +- **WHEN** the worktree holds `compose.yaml` +- **THEN** `siding up` uses it + +#### Scenario: No compose file +- **WHEN** the worktree holds no compose file and `--file` is not given +- **THEN** `siding up` fails with `ComposeFileNotFound` + +### Requirement: The compose file is never edited +siding MUST NOT write to the user's compose file, its `.env` or any other file of the user. It +SHALL write only `.siding/override.yml` and `.siding/.gitignore` (containing `*`) in the +worktree, so that `.siding/` never makes the worktree dirty. + +#### Scenario: After up +- **WHEN** `siding up` has run in a clean worktree +- **THEN** `git status --porcelain` prints nothing and the compose file is byte for byte unchanged + +### Requirement: Override +The override SHALL reset every service's `ports` (`ports: !reset []`), add every routed +service to the network `siding` next to its own networks, and add the `dev.siding.*` and +Traefik labels. The user's labels and settings MUST otherwise stay in effect. + +#### Scenario: A published port +- **WHEN** the compose file publishes `"5432:5432"` for `db` +- **THEN** after `siding up` no container of the stack publishes a port + +#### Scenario: Own networks +- **WHEN** `api` is on the user's networks `backend` and `default` +- **THEN** after `siding up` it is on `backend`, `default` and `siding` + +### Requirement: Up +`siding up` SHALL run `docker compose -p -f -f .siding/override.yml +up -d --remove-orphans`, followed by any arguments after `--`. It SHALL print the stack's +address, with `--json` as a `Stack` document, and fail with `ComposeFailed` and Compose's +message when Compose fails. + +#### Scenario: Passing arguments through +- **WHEN** `siding up -- --build web` runs +- **THEN** Compose builds and starts `web` and its dependencies + +#### Scenario: Compose fails +- **WHEN** an image of the compose file does not exist +- **THEN** `siding up` exits 1 with `ComposeFailed` and Compose's error in the message + +#### Scenario: Run twice +- **WHEN** `siding up` runs again in the same worktree +- **THEN** the same project is updated, and no second stack appears + +### Requirement: Up and wait +`siding up --wait [--timeout ]` SHALL start the stack as `up` does and then end exactly like +`siding status --wait`, with its document and exit code. + +#### Scenario: An agent starts and waits +- **WHEN** an agent runs `siding up --wait --json` and every service becomes ready +- **THEN** it prints one `Status` document with the verdict Ready and exits 0 + +### Requirement: Down +`siding down` SHALL remove the stack's containers and networks by project name, also when the +compose file is gone, and keep its volumes. It SHALL print what it removed, with `--json` as a +`Down` document, and with `--dry-run` remove nothing. + +#### Scenario: Data survives +- **WHEN** `siding down` and then `siding up` run +- **THEN** the stack's named volumes still hold their data + +#### Scenario: Dry run +- **WHEN** `siding down --dry-run --json` runs +- **THEN** it prints a `Down` document listing the containers and networks with `dryRun: true`, and everything still runs + +#### Scenario: Nothing running +- **WHEN** `siding down` runs in a worktree without a stack +- **THEN** it fails with `StackNotFound` + +### Requirement: Removing data keeps work +`siding down --volumes` SHALL also remove the stack's volumes, but MUST refuse with +`UnmergedBranch` when the branch is not merged into the default branch, and with +`DirtyWorktree` when `git status` lists changes, unless `--force` is given. + +#### Scenario: Unmerged branch +- **WHEN** `siding down --volumes` runs on a branch with commits that the default branch lacks +- **THEN** it fails with `UnmergedBranch`, removes nothing, and the hint names `--force` + +#### Scenario: Dirty worktree +- **WHEN** the worktree has uncommitted changes and `siding down --volumes` runs +- **THEN** it fails with `DirtyWorktree` and removes nothing + +#### Scenario: Forced +- **WHEN** `siding down --volumes --force` runs on an unmerged branch +- **THEN** the containers, networks and volumes of the stack are removed + +#### Scenario: No default branch +- **WHEN** siding finds neither `origin/HEAD` nor a local `main` or `master` +- **THEN** the branch counts as unmerged + +### Requirement: Logs +`siding logs [service…]` SHALL show the logs of the stack's services, following them with +`-f` and limited with `--tail `. With `--json` it SHALL print a `Logs` document of the last +lines per service; `--json` together with `-f` MUST fail with `InvalidArguments`. + +#### Scenario: One service +- **WHEN** `siding logs api --tail 5` runs +- **THEN** it prints the last 5 lines of `api` + +### Requirement: No .env values in JSON +Log lines in any document SHALL have every value defined in the worktree's `.env` and in the +services' `env_file`s, of 4 or more characters, replaced by `***`. + +#### Scenario: A password in a log line +- **WHEN** `.env` sets `DB_PASSWORD=not-a-real-token` and a service logs that value +- **THEN** the log line in the `Logs` and `Status` documents shows `***` instead diff --git a/openspec/specs/stack-list/spec.md b/openspec/specs/stack-list/spec.md new file mode 100644 index 0000000..3a7b27b --- /dev/null +++ b/openspec/specs/stack-list/spec.md @@ -0,0 +1,50 @@ +# stack-list Specification + +## Purpose +Defines `siding ps`, the one view of every stack on the machine: what runs, for which branch +and worktree, at which address, and what memory it takes right now. + +## Requirements + +### Requirement: Every stack on the machine +`siding ps` SHALL list every stack whose containers carry `dev.siding.repo`, found through the +Docker API from any directory, with repository, branch, state, address, memory in use and +worktree. It needs no git repository and no file of siding's. + +#### Scenario: Two worktrees +- **WHEN** stacks run for `shop` on `main` and on `feature/login` +- **THEN** `siding ps` lists both, each with its own address + +#### Scenario: A stack whose worktree is gone +- **WHEN** a worktree has been deleted while its stack still runs +- **THEN** `siding ps` still lists the stack, and marks the worktree as missing + +#### Scenario: Containers of other projects +- **WHEN** the daemon also runs containers without siding labels +- **THEN** `siding ps` does not list them + +### Requirement: Stack state in the list +The state in `ps` SHALL be `running` when every container of the stack runs, `partial` when +some do, and `stopped` when none does. `ps` MUST NOT probe the proxy; the full verdict is +`siding status`. + +#### Scenario: A stopped container +- **WHEN** one of three containers of a stack has exited +- **THEN** `siding ps` shows the stack as `partial` + +### Requirement: Memory in use +`ps` SHALL show each stack's memory in use, the sum over its running containers as the Docker +API reports it, and the total over all stacks. + +#### Scenario: Two stacks +- **WHEN** one stack uses 300 MiB and another 500 MiB +- **THEN** `siding ps` shows 300 MiB and 500 MiB for them and 800 MiB in total + +### Requirement: StackList document +With `--json`, `ps` SHALL print one `StackList` document with one entry per stack: project, +repository, branch, worktree, machine, agent, state, address, memory in bytes, and +`worktreeMissing` when the worktree is gone. + +#### Scenario: No stacks +- **WHEN** `siding ps --json` runs and no stack exists +- **THEN** it prints a `StackList` document without stacks and exits 0 diff --git a/openspec/specs/stack-status/spec.md b/openspec/specs/stack-status/spec.md new file mode 100644 index 0000000..b069b37 --- /dev/null +++ b/openspec/specs/stack-status/spec.md @@ -0,0 +1,84 @@ +# stack-status Specification + +## Purpose +Defines the verdict on a stack that agents and humans wait for: Ready only when it actually +runs and answers through the proxy, otherwise the service that failed, the reason, a hint and +the last lines of its log. + +## Requirements + +### Requirement: Verdict +`siding status` SHALL report one verdict for the worktree's stack: `Ready`, `Starting`, +`Failed` or `TimedOut`, the state of every service, and the stack's address. Without `--wait` +it SHALL report the current state once. + +#### Scenario: Status without wait +- **WHEN** `siding status` runs while a healthcheck is still starting +- **THEN** it reports `Starting` and exits 2 + +#### Scenario: No stack +- **WHEN** `siding status` runs in a worktree whose project has no containers +- **THEN** it fails with `StackNotFound` and the hint `siding up` + +### Requirement: Ready +A stack SHALL be `Ready` only when every service of the compose file has a container, every +service with a healthcheck is healthy, every other service is running or has exited 0, and +every service with a route path answers through the proxy. + +#### Scenario: Healthy and answering +- **WHEN** `db` is healthy, `web` with `dev.siding.route: "/"` runs and answers through the proxy, and `migrate` has exited 0 +- **THEN** the verdict is `Ready` + +#### Scenario: Running but not answering +- **WHEN** `web` with `dev.siding.route: "/"` runs but its process does not listen yet +- **THEN** the verdict is `Starting`, not `Ready` + +### Requirement: Answering through the proxy +A routed service SHALL count as answering when a request to its route through the proxy gets +an HTTP response from the service itself, of any status; a response the proxy produces on its +own (no router yet, 502, 503, 504) MUST NOT count. + +#### Scenario: The app answers 404 +- **WHEN** `api` with `dev.siding.route: "/api"` answers `/api` with 404 +- **THEN** `api` counts as answering + +#### Scenario: Bad gateway +- **WHEN** the proxy answers 502 for `/api` because nothing listens on the container port +- **THEN** `api` does not count as answering + +### Requirement: Failed +A stack SHALL be `Failed` as soon as a service has exited with a non-zero code, is unhealthy, +keeps restarting, or has no container; the failure MUST name the service, a reason +(`Exited`, `Unhealthy`, `Restarting`, `Missing`, `NotAnswering`), a hint, and the last 20 log +lines of that service. + +#### Scenario: A service exits +- **WHEN** `api` exits with code 1 after printing `cannot connect to db` +- **THEN** the verdict is `Failed` with service `api`, reason `Exited`, the exit code in the hint, and the log tail containing `cannot connect to db` + +#### Scenario: Unhealthy +- **WHEN** `db`'s healthcheck reports unhealthy +- **THEN** the verdict is `Failed` with reason `Unhealthy`, and the hint contains the last healthcheck output + +### Requirement: Waiting +`siding status --wait` SHALL check until the verdict is `Ready` or `Failed`, at most for +`--timeout` (default 3 minutes). When the time is up it SHALL report `TimedOut`, naming the +first service that is not ready, with reason, hint and log tail: `NotAnswering` for a routed +service that never answered, `NotReady` for one that is still starting. + +#### Scenario: Ready in time +- **WHEN** `siding status --wait --timeout 60s` runs and the stack becomes ready after 20 seconds +- **THEN** it reports `Ready` after about 20 seconds and exits 0 + +#### Scenario: Too slow +- **WHEN** `siding status --wait --timeout 10s` runs and `web` with route `/` never answers +- **THEN** it reports `TimedOut` with service `web`, reason `NotAnswering`, and exits 2 + +### Requirement: Status document +With `--json`, `status` SHALL print one `Status` document: the verdict, the stack (project, +repository, branch, worktree, address), every service (state, health, route, address, +answering), and for `Failed` and `TimedOut` the failure with service, reason, hint and log tail. + +#### Scenario: An agent reads the failure +- **WHEN** `siding status --wait --json` ends with `Failed` +- **THEN** the document holds `verdict: Failed` and a `failure` with `service`, `reason`, `hint` and `logs`, and the process exits 1