diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index f64d77b..ac98417 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -39,10 +39,13 @@ ], // The proxy (Traefik) listens on 8080 of the inner daemon; this forwards it to the host, so - // http://.localhost:8080 reaches a stack from the browser. + // http://.localhost:8080 reaches a stack from the browser. requireLocalPort: when 8080 + // is taken on the host, VS Code says so instead of forwarding to another port, which would + // make every address siding prints wrong. To use another port, change both numbers here and + // run siding setup --port with it. "forwardPorts": [8080], "portsAttributes": { - "8080": { "label": "siding proxy", "onAutoForward": "silent" } + "8080": { "label": "siding proxy", "onAutoForward": "silent", "requireLocalPort": true } }, // Worktrees with relative paths work both inside the container and on the host. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 067e518..3806cea 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,8 +1,8 @@ -# Level-1 checks and the Docker smoke test. Both run inside the devcontainer image, so CI uses -# exactly the tool versions pinned in .devcontainer/Dockerfile, and the smoke test runs against +# Level-1 checks and the Docker smoke tests. All run inside the devcontainer image, so CI uses +# exactly the tool versions pinned in .devcontainer/Dockerfile, and the smoke tests run against # the devcontainer's own Docker daemon, as locally. # -# `just test` runs on every push. The smoke test runs on pull requests and on main. +# `just test` runs on every push. The smoke tests run on pull requests and on main. name: ci on: @@ -39,14 +39,16 @@ jobs: export GOCACHE="$PWD/.cache/go/build" GOMODCACHE="$PWD/.cache/go/mod" just test - smoke-docker: + smoke: if: github.event_name == 'pull_request' || github.ref == 'refs/heads/main' runs-on: ubuntu-24.04-arm timeout-minutes: 30 steps: - uses: actions/checkout@v7 - - name: just smoke-docker (in the devcontainer) + - name: just smoke-docker and smoke-stacks (in the devcontainer) uses: devcontainers/ci@v0.3 with: push: never - runCmd: just smoke-docker + runCmd: | + just smoke-docker + just smoke-stacks diff --git a/.golangci.yml b/.golangci.yml index 6e4047f..fe09c66 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -1,5 +1,9 @@ version: "2" +# Also lint the end-to-end tests, which only build with the tag docker (just smoke-stacks). +run: + build-tags: [docker] + linters: default: standard enable: diff --git a/CLAUDE.md b/CLAUDE.md index d1df133..557edd3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -14,10 +14,11 @@ changing architecture. ## Status -Phase 0 done, waiting for approval (2026-10-28): devcontainer with Docker-in-Docker, OpenSpec -with the spec `dev-environment` and the draft changes `phase-1-…` to `phase-5-…` (v0.1–v0.5), -`siding version [--json]`, `just test`, `just smoke-docker`, `hack/nuke.sh`, CI. -Next: Phase 1 (`phase-1-stacks-and-routing`, v0.1). +Phase 0 approved (2026-10-28). Phase 1 (v0.1) approved and archived (2026-10-28, +`openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing`): `setup`, `up [--wait]`, +`status [--wait]`, `ps`, `logs`, `open`, `down [--volumes]`, `skill`, routing by host and path +through `siding-proxy`, `just smoke-stacks`. Next: Phase 2 (`phase-2-overview-and-budget`, +v0.2), starting with `/opsx:propose`. ## Working agreements @@ -52,8 +53,10 @@ Next: Phase 1 (`phase-1-stacks-and-routing`, v0.1). the last log lines of that service. - Agent and human see the same thing. Every reading command has `--json`; exit codes are 0 Ready, 1 Failed, 2 TimedOut, 3 NoCapacity. -- Never lose work. Removing a stack or worktree refuses when the branch is unmerged or the - worktree is dirty, unless forced; every destructive command supports `--dry-run`. +- Never lose work. Removing a stack's data (its volumes) or a worktree refuses when the branch + is unmerged or the worktree is dirty, unless forced; removing containers and networks loses + nothing and is not guarded (`siding down` keeps volumes, `down --volumes` is guarded). Every + destructive command supports `--dry-run`. - Only a human exposes anything to the internet. `siding share` is interactive only and has no `--yes`. - Standard building blocks (Docker Compose, Traefik, later Sablier) over custom code. @@ -66,11 +69,14 @@ Next: Phase 1 (`phase-1-stacks-and-routing`, v0.1). All development happens inside the devcontainer. It runs its own Docker daemon (Docker-in-Docker), which also hosts every stack siding's tests create. Nothing is installed on the host. Tool versions, Compose included, are pinned in `.devcontainer/Dockerfile`, the Docker engine in -`.devcontainer/devcontainer.json`. The proxy port 8080 is forwarded to the host, so -`http://.localhost:8080` reaches a stack from the browser. +`.devcontainer/devcontainer.json`. The proxy port 8080 is forwarded to the host: run +`bin/siding setup --port 8080` once in the container, and `http://.localhost:8080` +reaches a stack from the browser. The forward requires the host's 8080 (`requireLocalPort`): +when it is taken, VS Code says so instead of moving to another port. Free it, or change the +port in `devcontainer.json` and use the same one with `siding setup --port`. -CI runs `just test` on every push and `just smoke-docker` on pull requests and on `main`, both -inside the devcontainer image. +CI runs `just test` on every push and `just smoke-docker` and `just smoke-stacks` on pull +requests and on `main`, all inside the devcontainer image. | Command | Run from | Purpose | |---|---|---| @@ -78,6 +84,8 @@ inside the devcontainer image. | `just golden` | devcontainer | rewrite golden files after an intended change | | `just build` | devcontainer | build a static `bin/siding` (linux) | | `just smoke-docker` | devcontainer | inner daemon: compose project with two services and healthchecks, `ports: !reset []`, Traefik on `siding` routing `demo.main.localhost` at `/` and `/api` | +| `just smoke-stacks` | devcontainer | inner daemon: siding's own commands end to end (`internal/e2e`, build tag `docker`): setup, `up --wait`, routing, a Failed verdict with log tail, `ps`, `down`, the guard of `down --volumes`; also checks golden overrides with `docker compose config` | +| `bin/siding setup --port 8080` | devcontainer | the real `siding-proxy` on the forwarded port, to try siding by hand | | `openspec validate --all --strict` | devcontainer | check specs and active changes (part of `just test`) | | `openspec list` / `openspec list --specs` / `openspec view` | devcontainer | active changes, specs, dashboard | | `/opsx:explore`, `/opsx:propose`, `/opsx:apply`, `/opsx:archive` | Claude Code | the change workflow; skills in `.claude/skills/`, regenerated by `openspec update` | @@ -94,7 +102,9 @@ inside the devcontainer image. with `prune`. New host objects go into that spec, `hack/nuke.sh` and (for volumes) `devcontainer.json` together, in one change. - Tests create stacks only as Compose projects with the prefix `siding-test-` and remove them by - their exact name. The smoke test's proxy listens on 18080, never on 8080. + their exact name. The smoke tests' proxies listen on 18080, never on 8080; the end-to-end + tests run their own proxy `siding-test-e2e-proxy` on the network `siding-test-e2e`, so a + developer's `siding-proxy` is never touched. - No real secrets in the repo, in golden files or in logs; golden files may contain obvious fakes such as `not-a-real-token`. `.env` values never appear in `--json` output. - A worktree created with relative paths sets `extensions.relativeWorktrees` in `.git/config`, @@ -108,7 +118,9 @@ inside the devcontainer image. - Module `github.com/tweinmann/siding`, binary `siding`, API version `siding.dev/v1alpha1`. - Labels on every container, network and volume siding creates or adopts: `dev.siding.repo`, `dev.siding.branch`, `dev.siding.worktree` (absolute path), `dev.siding.machine` (stable - machine id), `dev.siding.agent` (optional). Service labels: `dev.siding.route`, + machine id); on containers also `dev.siding.agent` (optional; never on networks or volumes, + whose labels must not change between two `up`s). Service labels: `dev.siding.route` + (`/`, `/`, `false`), `dev.siding.port` (the HTTP port of a service with several), `dev.siding.seed` (`volume|script|none`). Volume label `dev.siding.shared`. - Compose project `-`, host `..localhost`, proxy `siding-proxy` on the network `siding`, override `.siding/override.yml`. diff --git a/README.md b/README.md index 8be9165..e2280f0 100644 --- a/README.md +++ b/README.md @@ -3,9 +3,78 @@ siding gives every git worktree its own Docker Compose stack, at its own `*.localhost` address, and tells you and your coding agents whether it actually runs and what it costs. -> **Pre-alpha.** Nothing here is usable yet: `siding version` is the only command. The plan is -> in [openspec/changes/](openspec/changes) (one change per phase), what siding does today in -> [openspec/specs/](openspec/specs). +> **Pre-alpha (v0.1).** Stacks per worktree, routing through one local proxy and the verdict +> of `siding status` work; expect rough edges and changes before v1. There are no releases +> yet: build it from source (see below). The plan is in [openspec/changes/](openspec/changes) +> (one change per phase), what siding does today in [openspec/specs/](openspec/specs). + +## Using siding + +siding needs Docker with Compose 2.24 or later, and git. Your compose file stays as it is. + +### Once per machine: the proxy + +```sh +siding setup # Traefik as siding-proxy on 127.0.0.1:80 +siding setup --port 8080 # if port 80 is taken, or inside siding's devcontainer +``` + +Every stack is reached through this one proxy at `http://..localhost`. Browsers +and curl resolve `*.localhost` to your machine by themselves; for other tools add the host to +`/etc/hosts`. Running `siding setup` again changes nothing unless the port differs. + +### In a worktree: start the stack + +```sh +siding up # e.g. shop on feature/login: http://shop.feature-login.localhost/ +siding up -- --build web # arguments after -- go to docker compose up +siding up --wait # and wait for the verdict, like siding status --wait +``` + +`up` runs your compose file as the Compose project `-` with one more file, +`.siding/override.yml`. That file resets every published port, so branches never fight over +ports, and puts siding's labels and routes on the services. `.siding/` ignores itself in git. + +- A service labelled `dev.siding.route: "/"` is the stack's root, `dev.siding.route: "/api"` + answers under that path (the path is passed on unchanged), `dev.siding.route: "false"` + hides a service. +- Every service with one container port (from `ports` or `expose`) also answers at + `http://...localhost`. With several ports, `dev.siding.port: "3000"` + names the HTTP one. +- Published ports are gone, also a database's. Reach it with + `docker compose -p - exec db psql …`. + +### Does it run? + +```sh +siding status # Ready, Starting, Failed or TimedOut, per service +siding status --wait # until Ready or Failed, at most --timeout (3m) +siding ps # every stack on this machine: state, address, memory +``` + +`Ready` means every service with a healthcheck is healthy and every service with a route +answers through the proxy. A failure names the service, the reason, a hint and its last log +lines. Exit codes: 0 Ready, 1 Failed, 2 TimedOut or still starting. Every command prints one +JSON document with `--json`, for scripts and agents. + +### Logs, address, and taking it down + +```sh +siding logs -f api # docker compose logs of the stack +siding open # the stack's address in the browser; siding open api for a service +siding down # remove containers and networks; volumes and data stay +siding down --volumes # also the data; refused while the branch is unmerged or the + # worktree dirty, unless --force; --dry-run lists it first +``` + +### For coding agents + +`siding skill` prints a Claude Code skill that tells an agent to run `siding up --wait --json` +after a change and to act on the verdict. siding writes no file itself; install it with + +```sh +mkdir -p .claude/skills/siding && siding skill > .claude/skills/siding/SKILL.md +``` ## Developing siding @@ -22,6 +91,14 @@ on your machine and siding's tests never touch your Docker containers. | `just golden` | rewrite golden files after an intended output change | | `just build` | build `bin/siding` | | `just smoke-docker` | Compose, `ports: !reset []` and Traefik routing against the container's own daemon | + | `just smoke-stacks` | siding's own commands end to end against the container's own daemon | + + To try siding by hand: `just build`, then `bin/siding setup --port 8080` (the port the + devcontainer forwards), and `bin/siding up --wait` in a project; its address, + `http://..localhost:8080`, opens in the browser on your machine. Port 8080 + must be free on your machine: VS Code reports it when it is not, rather than forwarding to + another port. Then free it, or change the port in `.devcontainer/devcontainer.json` and pass + the same one to `siding setup --port`. To remove everything siding development left on your Docker host, run `hack/nuke.sh` in a terminal on the host (`--dry-run` lists it first). diff --git a/cmd/siding/main.go b/cmd/siding/main.go index fcd5d16..778f220 100644 --- a/cmd/siding/main.go +++ b/cmd/siding/main.go @@ -3,14 +3,51 @@ package main import ( + "context" + "fmt" "os" + "os/signal" + "syscall" "github.com/tweinmann/siding/internal/cli" + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/probe" + "github.com/tweinmann/siding/internal/proxy" + "github.com/tweinmann/siding/internal/stack" ) // version is set at build time with -ldflags "-X main.version=". var version = "dev" func main() { - os.Exit(cli.Execute(cli.New(cli.Options{Build: cli.CurrentBuild(version)}), os.Args[1:], os.Stdout, os.Stderr)) + dir, err := os.Getwd() + if err != nil { + fmt.Fprintf(os.Stderr, "error: %v\n", err) + os.Exit(1) + } + home, err := os.UserHomeDir() + if err != nil { + fmt.Fprintf(os.Stderr, "error: %v\n", err) + os.Exit(1) + } + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) + opts := cli.Options{ + Build: cli.CurrentBuild(version), + Dir: dir, + Home: home, + Getenv: os.Getenv, + OSMachineID: stack.OSMachineID, + Git: git.Client{}, + Compose: &compose.CLI{}, + Docker: &docker.Client{}, + Probe: probe.HTTP{Header: stack.ServiceHeader}, + Browser: cli.SystemBrowser{Getenv: os.Getenv}, + Clock: cli.SystemClock{}, + Proxy: proxy.Default(), + } + code := cli.Execute(ctx, cli.New(opts), os.Args[1:], os.Stdout, os.Stderr) + stop() + os.Exit(code) } diff --git a/go.mod b/go.mod index 8e69a95..fb4aef7 100644 --- a/go.mod +++ b/go.mod @@ -2,9 +2,31 @@ module github.com/tweinmann/siding go 1.27.3 -require github.com/spf13/cobra v1.10.2 +require ( + github.com/containerd/errdefs v1.0.0 + github.com/moby/moby/api v1.56.1 + github.com/moby/moby/client v0.6.2 + github.com/spf13/cobra v1.10.2 +) require ( + github.com/Microsoft/go-winio v0.6.2 // indirect + github.com/containerd/errdefs/pkg v0.3.0 // indirect + github.com/distribution/reference v0.6.0 // indirect + github.com/docker/go-connections v0.8.2 // indirect + github.com/docker/go-units v0.5.0 // indirect + github.com/felixge/httpsnoop v1.0.4 // indirect + github.com/go-logr/logr v1.4.2 // indirect + github.com/go-logr/stdr v1.2.2 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/moby/docker-image-spec v1.3.1 // indirect + github.com/opencontainers/go-digest v1.0.0 // indirect + github.com/opencontainers/image-spec v1.1.1 // indirect github.com/spf13/pflag v1.0.9 // indirect + go.opentelemetry.io/auto/sdk v1.1.0 // indirect + go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 // indirect + go.opentelemetry.io/otel v1.35.0 // indirect + go.opentelemetry.io/otel/metric v1.35.0 // indirect + go.opentelemetry.io/otel/trace v1.35.0 // indirect + golang.org/x/sys v0.33.0 // indirect ) diff --git a/go.sum b/go.sum index a6ee3e0..e23e7ca 100644 --- a/go.sum +++ b/go.sum @@ -1,10 +1,71 @@ +github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY= +github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU= +github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI= +github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M= +github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE= +github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk= github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= +github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= +github.com/docker/go-connections v0.8.2 h1:9d9DEokx5W7qlr0shkFXIxEA7+TTD+llwzcykSlxMNc= +github.com/docker/go-connections v0.8.2/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q= +github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= +github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= +github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg= +github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= +github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY= +github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag= +github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0= +github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo= +github.com/moby/moby/api v1.56.1 h1:PpWkvVPB7Fr/No8w+TfyJ/I6rWZ2YPZhT3JorIg+06c= +github.com/moby/moby/api v1.56.1/go.mod h1:sZ+THbVWkjOmBPPfbnzdD/G1LuIexWhqlSHHPTDQ1Uk= +github.com/moby/moby/client v0.6.2 h1:ekSPlMccrZGcw6Scb8T9D39qE0mB+XupU/yS0zNw9N4= +github.com/moby/moby/client v0.6.2/go.mod h1:DGcnO7qmPJrqobx/ggQgQ/AUk+2wgVdQv7uxB0/mEsc= +github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U= +github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM= +github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040= +github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU= github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4= github.com/spf13/pflag v1.0.9 h1:9exaQaMOCwffKiiiYk6/BndUBv+iRViNW+4lEMi0PvY= github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= +github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA= +go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 h1:sbiXRNDSWJOTobXh5HyQKjq6wUC5tNybqjIqDpAY4CU= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0/go.mod h1:69uWxva0WgAA/4bu2Yy70SLDBwZXuQ6PbBpbsa5iZrQ= +go.opentelemetry.io/otel v1.35.0 h1:xKWKPxrxB6OtMCbmMY021CqC45J+3Onta9MqjhnusiQ= +go.opentelemetry.io/otel v1.35.0/go.mod h1:UEqy8Zp11hpkUrL73gSlELM0DupHoiq72dR+Zqel/+Y= +go.opentelemetry.io/otel/metric v1.35.0 h1:0znxYu2SNyuMSQT4Y9WDWej0VpcsxkuklLa4/siN90M= +go.opentelemetry.io/otel/metric v1.35.0/go.mod h1:nKVFgxBZ2fReX6IlyW28MgZojkoAkJGaE8CpgeAU3oE= +go.opentelemetry.io/otel/sdk v1.35.0 h1:iPctf8iprVySXSKJffSS79eOjl9pvxV9ZqOWT0QejKY= +go.opentelemetry.io/otel/sdk v1.35.0/go.mod h1:+ga1bZliga3DxJ3CQGg3updiaAJoNECOgJREo9KHGQg= +go.opentelemetry.io/otel/sdk/metric v1.35.0 h1:1RriWBmCKgkeHEhM7a2uMjMUfP7MsOF5JpUCaEqEI9o= +go.opentelemetry.io/otel/sdk/metric v1.35.0/go.mod h1:is6XYCUMpcKi+ZsOvfluY5YstFnhW0BidkR+gL+qN+w= +go.opentelemetry.io/otel/trace v1.35.0 h1:dPpEfJu1sDIqruz7BHFG3c7528f6ddfSWfFDVt/xgMs= +go.opentelemetry.io/otel/trace v1.35.0/go.mod h1:WUk7DtFp1Aw2MkvqGdwiXYDZZNvA/1J8o6xRXLrIkyc= go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= +golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw= +golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q= +gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA= +pgregory.net/rapid v1.2.0 h1:keKAYRcjm+e1F0oAuU5F5+YPAWcyxNNRK2wud503Gnk= +pgregory.net/rapid v1.2.0/go.mod h1:PY5XlDGj0+V1FCq0o192FdRhpKHGTRIWBgqjDBTrq04= diff --git a/hack/smoke/stacks.sh b/hack/smoke/stacks.sh new file mode 100755 index 0000000..5e3ed37 --- /dev/null +++ b/hack/smoke/stacks.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# End-to-end test of siding's own commands against the devcontainer's inner daemon: setup, +# up --wait, routing by host and path, a Failed verdict with its log tail, ps, down and the +# refusal of down --volumes. It runs the Go tests with the build tag docker: +# - internal/stack with -compose: docker compose accepts every golden override; +# - internal/e2e: the commands themselves, with the proxy siding-test-e2e-proxy on the +# network siding-test-e2e at port 18080 and projects named siding-test-*. +# Every object is removed by its exact name, also when a test fails. +set -euo pipefail +source "$(dirname "$0")/../lib.sh" +require_devcontainer +require_compose 2.24 + +cd "$(dirname "$0")/../.." +go test -count=1 ./internal/stack -compose +go test -count=1 -tags docker ./internal/e2e + +# Prove that nothing is left behind. +left="$( + docker ps -a --format '{{.Names}}' + docker network ls --format '{{.Name}}' + docker volume ls --format '{{.Name}}' +)" +if grep -q "^${SIDING_TEST_PREFIX}" <<<"$left"; then + die "objects named ${SIDING_TEST_PREFIX}* are left in the inner daemon: $(grep "^${SIDING_TEST_PREFIX}" <<<"$left" | tr '\n' ' ')" +fi +echo "smoke-stacks: ok" diff --git a/internal/cli/cli.go b/internal/cli/cli.go index 45ff1dd..4a33737 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -3,28 +3,103 @@ package cli import ( + "context" "errors" "fmt" "io" + "time" "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/probe" + "github.com/tweinmann/siding/internal/proxy" ) // Exit codes. They are part of the contract with agents, like the --json documents: 0 Ready, -// 1 Failed, 2 TimedOut, 3 NoCapacity. A command that fails with an Error document exits 1. +// 1 Failed, 2 TimedOut (or Starting without --wait), 3 NoCapacity. A command that fails with +// an Error document exits 1. const ( - exitOK = 0 - exitFailed = 1 + exitOK = 0 + exitFailed = 1 + exitTimedOut = 2 + // exitNoCapacity is reserved for the memory budget (v0.2). + exitNoCapacity = 3 + // exitInterrupted ends a command stopped by Ctrl-C or SIGTERM, without a message, as + // shells do. + exitInterrupted = 130 ) // Options holds everything a command needs from the machine it runs on. Tests pass their own, // so no package-level variable is a test seam and tests can run in parallel. type Options struct { Build BuildInfo + // Dir is the current directory. + Dir string + // Home is the user's home directory, for the cache under ~/.siding. + Home string + // Getenv reads an environment variable. + Getenv func(string) string + // OSMachineID returns the operating system's machine id, or "". + OSMachineID func() string + + Git Git + Compose Compose + Docker Docker + Probe Prober + Browser Browser + Clock Clock + // Proxy names siding-proxy and its network. + Proxy proxy.Config +} + +// Git answers questions about worktrees. +type Git interface { + Worktree(ctx context.Context, dir string) (git.Worktree, error) + Dirty(ctx context.Context, top string) (bool, error) + Merged(ctx context.Context, top, branch string) (merged bool, base string, err error) +} + +// Compose runs docker compose. +type Compose interface { + Config(ctx context.Context, inv compose.Invocation) ([]byte, error) + Run(ctx context.Context, inv compose.Invocation, args []string, stdout, stderr io.Writer) error +} + +// Docker reads state from the Docker Engine API. +type Docker interface { + Ping(ctx context.Context) error + Containers(ctx context.Context, labels map[string]string) ([]docker.Container, error) + Container(ctx context.Context, name string) (*docker.Container, error) + Networks(ctx context.Context, labels map[string]string) ([]docker.Network, error) + Volumes(ctx context.Context, labels map[string]string) ([]docker.Volume, error) + EnsureNetwork(ctx context.Context, name string, labels map[string]string) (bool, error) + Logs(ctx context.Context, id string, tail int) ([]string, error) + Memory(ctx context.Context, id string) (uint64, error) +} + +// Prober sends one request through the proxy. +type Prober interface { + Get(ctx context.Context, addr, host, path string) (probe.Response, error) +} + +// Browser opens a URL. +type Browser interface { + Open(url string) error +} + +// Clock is the time, for waiting. +type Clock interface { + Now() time.Time + Sleep(ctx context.Context, d time.Duration) error } // New returns the root command. func New(opts Options) *cobra.Command { + a := &app{opts: opts} root := &cobra.Command{ Use: "siding", Short: "One Docker Compose stack per git worktree, and one view of all of them", @@ -32,51 +107,84 @@ func New(opts Options) *cobra.Command { SilenceErrors: true, } root.CompletionOptions.DisableDefaultCmd = true + // An error is one line (and a hint); cobra's suggestions would add several. + root.DisableSuggestions = true root.SetFlagErrorFunc(func(_ *cobra.Command, err error) error { return &Error{Code: CodeInvalidArguments, Message: err.Error()} }) - root.AddCommand(newVersionCommand(opts.Build)) + root.AddCommand( + newVersionCommand(opts.Build), + newSetupCommand(a), + newUpCommand(a), + newStatusCommand(a), + newPsCommand(a), + newLogsCommand(a), + newOpenCommand(a), + newDownCommand(a), + newSkillCommand(), + ) return root } +// app gives commands the Options. +type app struct { + opts Options +} + // Execute runs root with args and returns the exit code. A failure is printed as an Error -// document on stdout when --json was given, and as one line on stderr otherwise. -func Execute(root *cobra.Command, args []string, stdout, stderr io.Writer) int { +// document on stdout when --json was given, and as one line (and a hint) on stderr otherwise. +func Execute(ctx context.Context, root *cobra.Command, args []string, stdout, stderr io.Writer) int { root.SetArgs(args) root.SetOut(stdout) root.SetErr(stderr) - cmd, err := root.ExecuteC() + cmd, err := root.ExecuteContextC(ctx) if err == nil { return exitOK } + var code exitCode + if errors.As(err, &code) { + return int(code) + } + if ctx.Err() != nil && errors.Is(err, ctx.Err()) { + return exitInterrupted + } var e *Error if !errors.As(err, &e) { - // Cobra reports an unknown command as a plain error. + // Cobra reports an unknown command as a plain error; anything else unexpected is + // still reported in the same shape. e = &Error{Code: CodeInvalidArguments, Message: err.Error()} + if cmd != nil && cmd != root { + e.Code = CodeInternal + } } if wantsJSON(cmd, args) { - if perr := printJSON(stdout, errorDocument{header: newHeader(kindError), Code: e.Code, Message: e.Message}); perr != nil { + doc := errorDocument{header: newHeader(kindError), Code: e.Code, Message: e.Message, Hint: e.Hint} + if perr := printJSON(stdout, doc); perr != nil { _, _ = fmt.Fprintf(stderr, "error: %s\n", e.Message) } } else { _, _ = fmt.Fprintf(stderr, "error: %s\n", e.Message) + if e.Hint != "" { + _, _ = fmt.Fprintf(stderr, "hint: %s\n", e.Hint) + } } return exitFailed } -// Error is a failure with a stable code, which agents may branch on. Codes are never renamed -// within an apiVersion. -type Error struct { - Code string - Message string -} +// exitCode ends a command that has already printed its result with this exit code. +type exitCode int -func (e *Error) Error() string { return e.Message } +func (c exitCode) Error() string { return fmt.Sprintf("exit code %d", int(c)) } -// Error codes. +// Error is a failure with a stable code, which agents may branch on. +type Error = fail.Error + +// Error codes Execute uses itself; the rest are in package fail. const ( // CodeInvalidArguments: unknown command, flag or argument. - CodeInvalidArguments = "InvalidArguments" + CodeInvalidArguments = fail.InvalidArguments + // CodeInternal: an unexpected error without a code of its own. + CodeInternal = fail.Internal ) // noArgs refuses positional arguments with CodeInvalidArguments. @@ -86,3 +194,21 @@ func noArgs(cmd *cobra.Command, args []string) error { } return nil } + +// SystemClock is the real time. +type SystemClock struct{} + +// Now returns the current time. +func (SystemClock) Now() time.Time { return time.Now() } + +// Sleep waits for d or until ctx is done. +func (SystemClock) Sleep(ctx context.Context, d time.Duration) error { + t := time.NewTimer(d) + defer t.Stop() + select { + case <-ctx.Done(): + return ctx.Err() + case <-t.C: + return nil + } +} diff --git a/internal/cli/cli_test.go b/internal/cli/cli_test.go index 9636e9e..a202261 100644 --- a/internal/cli/cli_test.go +++ b/internal/cli/cli_test.go @@ -2,9 +2,15 @@ package cli import ( "bytes" + "context" + "errors" "path/filepath" "testing" + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/testutil" ) @@ -17,7 +23,7 @@ var testBuild = BuildInfo{ func run(args ...string) (code int, stdout, stderr string) { var out, errOut bytes.Buffer - code = Execute(New(Options{Build: testBuild}), args, &out, &errOut) + code = Execute(context.Background(), New(Options{Build: testBuild}), args, &out, &errOut) return code, out.String(), errOut.String() } @@ -98,3 +104,81 @@ func TestErrorDocumentOnFlagError(t *testing.T) { t.Errorf("stdout:\n%s\nwant:\n%s", stdout, want) } } + +// Every error code reaches the user the same way: one line and a hint on stderr, or an Error +// document with code, message and hint on stdout. +func TestErrorCodes(t *testing.T) { + t.Parallel() + codes := []string{ + fail.InvalidArguments, fail.NotAGitRepository, fail.NoBranch, fail.ComposeFileNotFound, + fail.ComposeFailed, fail.DockerUnavailable, fail.ProxyNotRunning, fail.PortInUse, + fail.InvalidRoute, fail.AmbiguousPort, fail.SlugCollision, fail.StackNotFound, + fail.UnmergedBranch, fail.DirtyWorktree, fail.Internal, + } + for _, code := range codes { + t.Run(code, func(t *testing.T) { + t.Parallel() + root := &cobra.Command{Use: "siding", SilenceUsage: true, SilenceErrors: true} + var asJSON bool + sub := &cobra.Command{Use: "boom", RunE: func(*cobra.Command, []string) error { + return fail.New(code, "it broke").WithHint("try again") + }} + registerJSON(sub, &asJSON) + root.AddCommand(sub) + + var out, errOut bytes.Buffer + if got := Execute(context.Background(), root, []string{"boom"}, &out, &errOut); got != exitFailed { + t.Errorf("exit code %d", got) + } + if out.String() != "" || errOut.String() != "error: it broke\nhint: try again\n" { + t.Errorf("text: stdout %q, stderr %q", out.String(), errOut.String()) + } + + out.Reset() + errOut.Reset() + root.SetArgs(nil) + Execute(context.Background(), root, []string{"boom", "--json"}, &out, &errOut) + want := "{\n \"apiVersion\": \"siding.dev/v1alpha1\",\n \"kind\": \"Error\",\n \"code\": \"" + code + + "\",\n \"message\": \"it broke\",\n \"hint\": \"try again\"\n}\n" + if out.String() != want || errOut.String() != "" { + t.Errorf("json: stdout %s, stderr %q", out.String(), errOut.String()) + } + }) + } +} + +// A command that printed a verdict ends with its exit code and nothing more. +func TestExitCodes(t *testing.T) { + t.Parallel() + for _, code := range []int{exitFailed, exitTimedOut, exitNoCapacity} { + root := &cobra.Command{Use: "siding", SilenceUsage: true, SilenceErrors: true} + root.AddCommand(&cobra.Command{Use: "verdict", RunE: func(*cobra.Command, []string) error { return exitCode(code) }}) + var out, errOut bytes.Buffer + if got := Execute(context.Background(), root, []string{"verdict"}, &out, &errOut); got != code || out.Len()+errOut.Len() != 0 { + t.Errorf("exit code %d (want %d), output %q %q", got, code, out.String(), errOut.String()) + } + } + // An error without a code is still reported in the same shape. + root := &cobra.Command{Use: "siding", SilenceUsage: true, SilenceErrors: true} + root.AddCommand(&cobra.Command{Use: "oops", RunE: func(*cobra.Command, []string) error { return errors.New("disk full") }}) + var out, errOut bytes.Buffer + if got := Execute(context.Background(), root, []string{"oops"}, &out, &errOut); got != exitFailed || errOut.String() != "error: disk full\n" { + t.Errorf("exit code %d, stderr %q", got, errOut.String()) + } +} + +// Ctrl-C ends a command quietly with 130, e.g. logs --follow. +func TestInterrupted(t *testing.T) { + t.Parallel() + ctx, cancel := context.WithCancel(context.Background()) + root := &cobra.Command{Use: "siding", SilenceUsage: true, SilenceErrors: true} + root.AddCommand(&cobra.Command{Use: "follow", RunE: func(cmd *cobra.Command, _ []string) error { + cancel() + <-cmd.Context().Done() + return cmd.Context().Err() + }}) + var out, errOut bytes.Buffer + if got := Execute(ctx, root, []string{"follow"}, &out, &errOut); got != exitInterrupted || out.Len()+errOut.Len() != 0 { + t.Errorf("exit code %d, output %q %q", got, out.String(), errOut.String()) + } +} diff --git a/internal/cli/down.go b/internal/cli/down.go new file mode 100644 index 0000000..484e245 --- /dev/null +++ b/internal/cli/down.go @@ -0,0 +1,159 @@ +package cli + +import ( + "context" + "fmt" + "sort" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" +) + +type downFlags struct { + volumes, force, dryRun bool +} + +func newDownCommand(a *app) *cobra.Command { + var asJSON bool + var f downFlags + cmd := &cobra.Command{ + Use: "down", + Short: "Remove the worktree's stack; its volumes stay unless --volumes", + Long: `Remove the containers and networks of the worktree's stack. Its volumes, and with them +its data, stay. --volumes removes them too, but refuses while the branch is not merged into the +default branch or the worktree has uncommitted changes, unless --force.`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + doc, err := a.down(cmd, f) + if err != nil { + return err + } + if asJSON { + return printJSON(cmd.OutOrStdout(), doc) + } + return printDownText(cmd, doc, f.volumes) + }, + } + cmd.Flags().BoolVar(&f.volumes, "volumes", false, "also remove the stack's volumes and their data") + cmd.Flags().BoolVar(&f.force, "force", false, "remove the volumes even if the branch is unmerged or the worktree dirty") + cmd.Flags().BoolVar(&f.dryRun, "dry-run", false, "list what would be removed, remove nothing") + registerJSON(cmd, &asJSON) + return cmd +} + +func (a *app) down(cmd *cobra.Command, f downFlags) (downDocument, error) { + ctx := cmd.Context() + st, err := a.findStack(ctx) + if err != nil { + return downDocument{}, err + } + if f.volumes && !f.force { + if err := a.keepsWork(ctx, st); err != nil { + return downDocument{}, err + } + } + doc, err := a.downPlan(ctx, st, f) + if err != nil { + return downDocument{}, err + } + if f.dryRun { + return doc, nil + } + args := []string{"down", "--remove-orphans"} + if f.volumes { + args = append(args, "--volumes") + } + // By project name, without files: it works even when the compose file is gone. + inv := compose.Invocation{Dir: st.wt.Top, Project: st.project} + if err := a.opts.Compose.Run(ctx, inv, args, cmd.ErrOrStderr(), cmd.ErrOrStderr()); err != nil { + return downDocument{}, err + } + return doc, nil +} + +// keepsWork refuses to remove data while the worktree has changes or the branch is unmerged. +func (a *app) keepsWork(ctx context.Context, st existing) error { + dirty, err := a.opts.Git.Dirty(ctx, st.wt.Top) + if err != nil { + return err + } + if dirty { + return fail.New(fail.DirtyWorktree, "the worktree %s has uncommitted changes; its volumes stay", st.wt.Top). + WithHint("commit or stash them first, or remove the data anyway with --force") + } + merged, base, err := a.opts.Git.Merged(ctx, st.wt.Top, st.wt.Branch) + if err != nil { + return err + } + if !merged { + into := "no default branch was found (origin/HEAD, main or master)" + if base != "" { + into = "it is not merged into " + strings.TrimPrefix(strings.TrimPrefix(base, "refs/remotes/"), "refs/heads/") + } + return fail.New(fail.UnmergedBranch, "the branch %s may hold unmerged work: %s; its volumes stay", st.wt.Branch, into). + WithHint("merge it first, or remove the data anyway with --force (a squash merge also needs --force)") + } + return nil +} + +// downPlan lists what down removes. +func (a *app) downPlan(ctx context.Context, st existing, f downFlags) (downDocument, error) { + doc := downDocument{header: newHeader(kindDown), Stack: st.ref(a.shownPort(ctx)), DryRun: f.dryRun} + byProject := map[string]string{stack.LabelProject: st.project} + cs, err := a.opts.Docker.Containers(ctx, byProject) + if err != nil { + return doc, err + } + for _, c := range cs { + doc.Containers = append(doc.Containers, c.Name) + } + ns, err := a.opts.Docker.Networks(ctx, byProject) + if err != nil { + return doc, err + } + for _, n := range ns { + doc.Networks = append(doc.Networks, n.Name) + } + if f.volumes { + vs, err := a.opts.Docker.Volumes(ctx, byProject) + if err != nil { + return doc, err + } + for _, v := range vs { + doc.Volumes = append(doc.Volumes, v.Name) + } + } + sort.Strings(doc.Containers) + sort.Strings(doc.Networks) + sort.Strings(doc.Volumes) + return doc, nil +} + +func printDownText(cmd *cobra.Command, doc downDocument, volumes bool) error { + var b strings.Builder + verb := "Removed" + if doc.DryRun { + verb = "Would remove" + } + fmt.Fprintf(&b, "%s from %s:\n", verb, doc.Stack.Project) + for _, list := range []struct { + kind string + names []string + }{{"container", doc.Containers}, {"network", doc.Networks}, {"volume", doc.Volumes}} { + for _, n := range list.names { + fmt.Fprintf(&b, " %s %s\n", list.kind, n) + } + } + if !volumes { + b.WriteString("Volumes stay; siding down --volumes removes them.\n") + } + if doc.DryRun { + b.WriteString("Dry run: nothing was removed.\n") + } + _, err := fmt.Fprint(cmd.OutOrStdout(), b.String()) + return err +} diff --git a/internal/cli/down_test.go b/internal/cli/down_test.go new file mode 100644 index 0000000..cbeee1e --- /dev/null +++ b/internal/cli/down_test.go @@ -0,0 +1,118 @@ +package cli + +import ( + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" + "github.com/tweinmann/siding/internal/testutil" +) + +// downEnv is a stack on branch with a network and a volume. +func downEnv(t *testing.T, branch string) *env { + t.Helper() + e := newEnv(t) + if branch != "main" { + e.git("switch", "-q", "-c", branch) + e.git("commit", "-q", "--allow-empty", "-m", "work") + } + id := e.id() + id.Branch = branch + e.proxyUp(80) + e.stackUp(id, "db", "web") + project := map[string]string{stack.LabelProject: id.Project()} + e.docker.networks = []docker.Network{{Name: id.Project() + "_default", Labels: project}, {Name: "siding", Labels: map[string]string{}}} + e.docker.volumes = []docker.Volume{{Name: id.Project() + "_data", Labels: project}, {Name: "shared-cache", Labels: map[string]string{}}} + return e +} + +func TestDown(t *testing.T) { + t.Parallel() + + t.Run("keeps the volumes", func(t *testing.T) { + t.Parallel() + e := downEnv(t, "main") + code, stdout, _ := e.run("down", "--json") + if code != exitOK { + t.Fatalf("exit %d: %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "down.json"), []byte(stdout)) + runs := e.compose.recorded() + if len(runs) != 1 || runs[0].inv.Project != "shop-main" || len(runs[0].inv.Files) != 0 || + !slices.Equal(runs[0].args, []string{"down", "--remove-orphans"}) { + t.Errorf("compose runs %+v", runs) + } + }) + + t.Run("dry run", func(t *testing.T) { + t.Parallel() + e := downEnv(t, "main") + code, stdout, _ := e.run("down", "--volumes", "--dry-run", "--json") + if code != exitOK || len(e.compose.recorded()) != 0 { + t.Fatalf("exit %d, compose runs %d", code, len(e.compose.recorded())) + } + testutil.Golden(t, filepath.Join("testdata", "json", "down-dryrun.json"), []byte(stdout)) + + _, text, _ := e.run("down", "--dry-run") + want := "Would remove from shop-main:\n container shop-main-db-1\n container shop-main-web-1\n network shop-main_default\n" + + "Volumes stay; siding down --volumes removes them.\nDry run: nothing was removed.\n" + if text != want { + t.Errorf("text:\n%s\nwant:\n%s", text, want) + } + }) + + t.Run("volumes of a merged, clean branch", func(t *testing.T) { + t.Parallel() + e := downEnv(t, "main") + if code, stdout, _ := e.run("down", "--volumes", "--json"); code != exitOK || !strings.Contains(stdout, "shop-main_data") { + t.Fatalf("exit %d: %s", code, stdout) + } + if runs := e.compose.recorded(); len(runs) != 1 || !slices.Equal(runs[0].args, []string{"down", "--remove-orphans", "--volumes"}) { + t.Errorf("compose runs %+v", runs) + } + }) + + for _, tt := range []struct { + name string + branch string + dirty bool + code string + }{ + {name: "unmerged branch", branch: "wip", code: fail.UnmergedBranch}, + {name: "dirty worktree", branch: "main", dirty: true, code: fail.DirtyWorktree}, + } { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + e := downEnv(t, tt.branch) + if tt.dirty { + e.write("notes.txt", "unsaved") + } + code, _, stderr := e.run("down", "--volumes") + if code != exitFailed || !strings.Contains(stderr, "--force") || len(e.compose.recorded()) != 0 { + t.Fatalf("exit %d, stderr %q, compose runs %d", code, stderr, len(e.compose.recorded())) + } + if _, stdout, _ := e.run("down", "--volumes", "--json"); !strings.Contains(stdout, `"code": "`+tt.code+`"`) { + t.Errorf("stdout %s, want code %s", stdout, tt.code) + } + // Without --volumes nothing is lost, so there is no guard. + if code, _, _ := e.run("down"); code != exitOK { + t.Errorf("down without --volumes: exit %d", code) + } + if code, _, _ := e.run("down", "--volumes", "--force"); code != exitOK { + t.Errorf("forced: exit %d", code) + } + }) + } + + t.Run("no stack", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + if code, stdout, _ := e.run("down", "--json"); code != exitFailed || !strings.Contains(stdout, `"code": "StackNotFound"`) { + t.Errorf("exit %d, stdout %s", code, stdout) + } + }) +} diff --git a/internal/cli/env_test.go b/internal/cli/env_test.go new file mode 100644 index 0000000..3cd23fb --- /dev/null +++ b/internal/cli/env_test.go @@ -0,0 +1,373 @@ +package cli + +import ( + "bytes" + "context" + "io" + "maps" + "os" + "os/exec" + "path/filepath" + "strings" + "sync" + "testing" + "time" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/probe" + "github.com/tweinmann/siding/internal/proxy" + "github.com/tweinmann/siding/internal/stack" +) + +// testConfig is what docker compose config would print for testCompose: web on / (one port), +// api on /api (a port label picks 8080 of two), db with no port, and mail with one exposed +// port but no route label. +const testConfig = `{ + "name": "shop-main", + "services": { + "api": {"expose": ["8080", "9229"], "labels": {"dev.siding.route": "/api", "dev.siding.port": "8080"}, + "networks": {"default": null}, "environment": {"API_TOKEN": "not-a-real-api-token"}}, + "db": {"networks": {"default": null}, "environment": {"POSTGRES_PASSWORD": "not-a-real-token"}}, + "mail": {"expose": ["8025"], "networks": {"default": null}}, + "web": {"ports": [{"target": 3000, "published": "3000", "protocol": "tcp"}], "labels": {"dev.siding.route": "/"}, + "networks": {"default": null}} + }, + "networks": {"default": {"name": "shop-main_default"}}, + "volumes": {"data": {"name": "shop-main_data"}} +}` + +// env is one test's machine: a real git repository /shop on main with a compose file, +// and fakes for everything else. +type env struct { + t *testing.T + base string // parent of the repository, shown as /src in golden files + repo string + opts Options + docker *fakeDocker + compose *fakeCompose + probe *fakeProbe + browser *fakeBrowser + clock *fakeClock + envVars map[string]string +} + +func newEnv(t *testing.T) *env { + t.Helper() + // The real path: git reports it, and on macOS the temporary directory is a symlink. + base, err := filepath.EvalSymlinks(t.TempDir()) + if err != nil { + t.Fatal(err) + } + e := &env{ + t: t, + base: base, + repo: filepath.Join(base, "shop"), + docker: &fakeDocker{logs: map[string][]string{}, memory: map[string]uint64{}}, + compose: &fakeCompose{config: testConfig}, + probe: &fakeProbe{answers: map[string]probe.Response{}}, + browser: &fakeBrowser{}, + clock: &fakeClock{now: time.Date(2026, 10, 28, 12, 0, 0, 0, time.UTC)}, + envVars: map[string]string{}, + } + if err := os.MkdirAll(e.repo, 0o755); err != nil { + t.Fatal(err) + } + e.git("init", "-q", "-b", "main") + e.write("compose.yaml", "services: {}\n") + e.write(".env", "POSTGRES_PASSWORD=not-a-real-token\n") + e.git("add", ".") + e.git("commit", "-q", "-m", "first") + e.opts = Options{ + Build: testBuild, + Dir: e.repo, + Home: filepath.Join(base, "home"), + Getenv: func(k string) string { return e.envVars[k] }, + OSMachineID: func() string { return "not-a-real-machine-id" }, + Git: git.Client{}, + Compose: e.compose, + Docker: e.docker, + Probe: e.probe, + Browser: e.browser, + Clock: e.clock, + Proxy: proxy.Default(), + } + return e +} + +func (e *env) git(args ...string) { + e.t.Helper() + cmd := exec.Command("git", append([]string{"-C", e.repo}, args...)...) + cmd.Env = append(os.Environ(), "GIT_CONFIG_GLOBAL=/dev/null", "GIT_CONFIG_NOSYSTEM=1", + "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@example.com", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@example.com") + if out, err := cmd.CombinedOutput(); err != nil { + e.t.Fatalf("git %v: %v\n%s", args, err, out) + } +} + +func (e *env) write(name, content string) { + e.t.Helper() + path := filepath.Join(e.repo, name) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + e.t.Fatal(err) + } + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + e.t.Fatal(err) + } +} + +// run runs siding with args and returns exit code, stdout and stderr, with the temporary +// directory shown as /src. +func (e *env) run(args ...string) (code int, stdout, stderr string) { + var out, errOut bytes.Buffer + code = Execute(context.Background(), New(e.opts), args, &out, &errOut) + return code, e.clean(out.String()), e.clean(errOut.String()) +} + +func (e *env) clean(s string) string { return strings.ReplaceAll(s, e.base, "/src") } + +// id is the identity siding gives the test repository. +func (e *env) id() stack.Identity { + machine, err := stack.MachineID("not-a-real-machine-id", "") + if err != nil { + e.t.Fatal(err) + } + return stack.Identity{Repo: "shop", Branch: "main", Worktree: e.repo, Machine: machine} +} + +// proxyUp makes siding-proxy run on port. +func (e *env) proxyUp(port int) { + e.docker.mu.Lock() + defer e.docker.mu.Unlock() + e.docker.proxy = &docker.Container{ID: "proxy", Name: "siding-proxy", Image: proxy.Image, State: "running", + Labels: map[string]string{}, Ports: []docker.Port{{HostIP: "127.0.0.1", HostPort: port, ContainerPort: 80}}} +} + +// stackUp adds one container per service of the test stack, running and, where routed, +// answering through the proxy. +func (e *env) stackUp(id stack.Identity, services ...string) { + for _, s := range services { + e.docker.add(container(id, s, "running", "", 0)) + } + e.probe.set("/", id.Project()+"/web", 200) + e.probe.set("/api", id.Project()+"/api", 404) +} + +func container(id stack.Identity, service, state, health string, exit int) docker.Container { + labels := id.ContainerLabels() + labels[stack.LabelProject] = id.Project() + labels[stack.LabelService] = service + return docker.Container{ID: id.Project() + "-" + service + "-1", Name: id.Project() + "-" + service + "-1", + Labels: labels, State: state, Health: health, ExitCode: exit} +} + +type fakeDocker struct { + mu sync.Mutex + pingErr error + proxy *docker.Container + containers []docker.Container + networks []docker.Network + volumes []docker.Volume + logs map[string][]string + memory map[string]uint64 + ensured []string +} + +func (d *fakeDocker) add(c docker.Container) { + d.mu.Lock() + defer d.mu.Unlock() + d.containers = append(d.containers, c) +} + +// update changes the container of a service. +func (d *fakeDocker) update(service string, f func(*docker.Container)) { + d.mu.Lock() + defer d.mu.Unlock() + for i := range d.containers { + if d.containers[i].Labels[stack.LabelService] == service { + f(&d.containers[i]) + } + } +} + +func matches(have, want map[string]string) bool { + for k, v := range want { + got, ok := have[k] + if !ok || (v != "" && got != v) { + return false + } + } + return true +} + +func (d *fakeDocker) Ping(context.Context) error { return d.pingErr } + +func (d *fakeDocker) Containers(_ context.Context, labels map[string]string) ([]docker.Container, error) { + d.mu.Lock() + defer d.mu.Unlock() + var out []docker.Container + for _, c := range d.containers { + if matches(c.Labels, labels) { + c.Labels = maps.Clone(c.Labels) + out = append(out, c) + } + } + return out, nil +} + +func (d *fakeDocker) Container(_ context.Context, name string) (*docker.Container, error) { + d.mu.Lock() + defer d.mu.Unlock() + if d.proxy != nil && d.proxy.Name == name { + c := *d.proxy + return &c, nil + } + return nil, nil +} + +func (d *fakeDocker) Networks(_ context.Context, labels map[string]string) ([]docker.Network, error) { + d.mu.Lock() + defer d.mu.Unlock() + var out []docker.Network + for _, n := range d.networks { + if matches(n.Labels, labels) { + out = append(out, n) + } + } + return out, nil +} + +func (d *fakeDocker) Volumes(_ context.Context, labels map[string]string) ([]docker.Volume, error) { + d.mu.Lock() + defer d.mu.Unlock() + var out []docker.Volume + for _, v := range d.volumes { + if matches(v.Labels, labels) { + out = append(out, v) + } + } + return out, nil +} + +func (d *fakeDocker) EnsureNetwork(_ context.Context, name string, _ map[string]string) (bool, error) { + d.mu.Lock() + defer d.mu.Unlock() + d.ensured = append(d.ensured, name) + return true, nil +} + +func (d *fakeDocker) Logs(_ context.Context, id string, tail int) ([]string, error) { + d.mu.Lock() + defer d.mu.Unlock() + lines := d.logs[id] + if len(lines) > tail { + lines = lines[len(lines)-tail:] + } + return lines, nil +} + +func (d *fakeDocker) Memory(_ context.Context, id string) (uint64, error) { + d.mu.Lock() + defer d.mu.Unlock() + return d.memory[id], nil +} + +type composeRun struct { + inv compose.Invocation + args []string +} + +type fakeCompose struct { + mu sync.Mutex + config string + runs []composeRun + runErr error + // onRun, if set, runs after each recorded run, e.g. to start containers. + onRun func(composeRun) +} + +func (c *fakeCompose) Config(context.Context, compose.Invocation) ([]byte, error) { + return []byte(c.config), nil +} + +func (c *fakeCompose) Run(_ context.Context, inv compose.Invocation, args []string, _, _ io.Writer) error { + c.mu.Lock() + r := composeRun{inv: inv, args: args} + c.runs = append(c.runs, r) + onRun, err := c.onRun, c.runErr + c.mu.Unlock() + if err != nil { + return err + } + if onRun != nil { + onRun(r) + } + return nil +} + +func (c *fakeCompose) recorded() []composeRun { + c.mu.Lock() + defer c.mu.Unlock() + return append([]composeRun(nil), c.runs...) +} + +type fakeProbe struct { + mu sync.Mutex + answers map[string]probe.Response // by path + asked []string +} + +func (p *fakeProbe) set(path, service string, status int) { + p.mu.Lock() + defer p.mu.Unlock() + p.answers[path] = probe.Response{Status: status, Service: service} +} + +func (p *fakeProbe) Get(_ context.Context, addr, host, path string) (probe.Response, error) { + p.mu.Lock() + defer p.mu.Unlock() + p.asked = append(p.asked, addr+" "+host+path) + if r, ok := p.answers[path]; ok { + return r, nil + } + return probe.Response{Status: 404}, nil // like Traefik without a router +} + +type fakeBrowser struct { + mu sync.Mutex + opened []string +} + +func (b *fakeBrowser) Open(url string) error { + b.mu.Lock() + defer b.mu.Unlock() + b.opened = append(b.opened, url) + return nil +} + +// fakeClock advances only when slept on, and runs onSleep after each sleep. +type fakeClock struct { + mu sync.Mutex + now time.Time + slept int + onSleep func(slept int) +} + +func (c *fakeClock) Now() time.Time { + c.mu.Lock() + defer c.mu.Unlock() + return c.now +} + +func (c *fakeClock) Sleep(_ context.Context, d time.Duration) error { + c.mu.Lock() + c.now = c.now.Add(d) + c.slept++ + n, f := c.slept, c.onSleep + c.mu.Unlock() + if f != nil { + f(n) + } + return nil +} diff --git a/internal/cli/json.go b/internal/cli/json.go index 02b8455..be95652 100644 --- a/internal/cli/json.go +++ b/internal/cli/json.go @@ -18,8 +18,16 @@ import ( const documentAPIVersion = "siding.dev/v1alpha1" const ( - kindError = "Error" - kindVersion = "Version" + kindAddress = "Address" + kindDown = "Down" + kindError = "Error" + kindLogs = "Logs" + kindSetup = "Setup" + kindSkill = "Skill" + kindStack = "Stack" + kindStackList = "StackList" + kindStatus = "Status" + kindVersion = "Version" ) // jsonFlag is the name of the flag, looked up again by Execute to report an error as a document. @@ -58,6 +66,7 @@ func wantsJSON(cmd *cobra.Command, args []string) bool { func printJSON(w io.Writer, doc any) error { enc := json.NewEncoder(w) enc.SetIndent("", " ") + enc.SetEscapeHTML(false) return enc.Encode(doc) } @@ -66,6 +75,7 @@ type errorDocument struct { header Code string `json:"code"` Message string `json:"message"` + Hint string `json:"hint,omitempty"` } type versionDocument struct { @@ -75,3 +85,127 @@ type versionDocument struct { GoVersion string `json:"goVersion"` Platform string `json:"platform"` } + +// stackRef names a stack in other documents. +type stackRef struct { + Project string `json:"project"` + Repo string `json:"repo"` + Branch string `json:"branch"` + Worktree string `json:"worktree"` + // URL is the stack's address: its host on the proxy's port. + URL string `json:"url,omitempty"` +} + +// setupDocument is what setup prints. +type setupDocument struct { + header + Container string `json:"container"` + Network string `json:"network"` + Image string `json:"image"` + Port int `json:"port"` + URL string `json:"url"` + // State is created, updated or unchanged. + State string `json:"state"` +} + +// stackDocument is what up prints without --wait. +type stackDocument struct { + header + Stack stackRef `json:"stack"` + Services []serviceAddress `json:"services,omitempty"` +} + +type serviceAddress struct { + Name string `json:"name"` + // URL is the service host. + URL string `json:"url"` + // Route is the path on the stack's host, if any. + Route string `json:"route,omitempty"` + Port int `json:"port"` +} + +// statusDocument is the verdict of status and up --wait. +type statusDocument struct { + header + // Verdict is Ready, Starting, Failed or TimedOut. + Verdict string `json:"verdict"` + Stack stackRef `json:"stack"` + Services []serviceStatus `json:"services,omitempty"` + Failure *failureDocument `json:"failure,omitempty"` +} + +type serviceStatus struct { + Name string `json:"name"` + State string `json:"state"` + Health string `json:"health,omitempty"` + Route string `json:"route,omitempty"` + URL string `json:"url,omitempty"` + // Answering is set for services with a route path. + Answering *bool `json:"answering,omitempty"` + Ready bool `json:"ready"` +} + +type failureDocument struct { + Service string `json:"service"` + // Reason is Exited, Unhealthy, Restarting, Missing, NotAnswering or NotReady. + Reason string `json:"reason"` + Hint string `json:"hint"` + Logs []string `json:"logs,omitempty"` +} + +// stackListDocument is what ps prints. +type stackListDocument struct { + header + Stacks []stackEntry `json:"stacks,omitempty"` + MemoryBytes uint64 `json:"memoryBytes,omitempty"` +} + +type stackEntry struct { + Project string `json:"project"` + Repo string `json:"repo"` + Branch string `json:"branch"` + Worktree string `json:"worktree"` + Machine string `json:"machine,omitempty"` + Agent string `json:"agent,omitempty"` + // State is running, partial or stopped. + State string `json:"state"` + URL string `json:"url"` + MemoryBytes uint64 `json:"memoryBytes,omitempty"` + WorktreeMissing bool `json:"worktreeMissing,omitempty"` +} + +// logsDocument is what logs --json prints. +type logsDocument struct { + header + Stack stackRef `json:"stack"` + Services []serviceLogs `json:"services,omitempty"` +} + +type serviceLogs struct { + Name string `json:"name"` + Lines []string `json:"lines,omitempty"` +} + +// addressDocument is what open --json prints. +type addressDocument struct { + header + URL string `json:"url"` + Service string `json:"service,omitempty"` +} + +// downDocument is what down prints. +type downDocument struct { + header + Stack stackRef `json:"stack"` + DryRun bool `json:"dryRun,omitempty"` + Containers []string `json:"containers,omitempty"` + Networks []string `json:"networks,omitempty"` + Volumes []string `json:"volumes,omitempty"` +} + +// skillDocument is what skill --json prints. +type skillDocument struct { + header + Name string `json:"name"` + Text string `json:"text"` +} diff --git a/internal/cli/logs.go b/internal/cli/logs.go new file mode 100644 index 0000000..5d531fa --- /dev/null +++ b/internal/cli/logs.go @@ -0,0 +1,94 @@ +package cli + +import ( + "slices" + "strconv" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" +) + +// defaultJSONTail is how many lines per service logs --json prints without --tail. +const defaultJSONTail = 100 + +func newLogsCommand(a *app) *cobra.Command { + var asJSON, follow bool + var tail int + var file string + cmd := &cobra.Command{ + Use: "logs [service...]", + Short: "Show the logs of the worktree's stack", + Long: `Show the logs of the worktree's stack, of all services or of the ones named. With +--json it prints the last lines per service (100 without --tail) as one document, with the +values of .env and of the services' environment replaced by ***.`, + RunE: func(cmd *cobra.Command, services []string) error { + if asJSON && follow { + return fail.New(fail.InvalidArguments, "--json and --follow cannot be combined: --json prints one document") + } + ctx := cmd.Context() + st, err := a.findStack(ctx) + if err != nil { + return err + } + if !asJSON { + args := []string{"logs"} + if follow { + args = append(args, "--follow") + } + if tail > 0 { + args = append(args, "--tail", strconv.Itoa(tail)) + } + inv := compose.Invocation{Dir: st.wt.Top, Project: st.project} + return a.opts.Compose.Run(ctx, inv, append(args, services...), cmd.OutOrStdout(), cmd.ErrOrStderr()) + } + + if tail <= 0 { + tail = defaultJSONTail + } + _, p, err := a.loadProject(ctx, st.wt.Top, file, st.project) + if err != nil { + return err + } + r, err := redactor(st.wt.Top, p) + if err != nil { + return err + } + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelProject: st.project}) + if err != nil { + return err + } + grouped := byService(cs) + names := services + if len(names) == 0 { + names = p.ServiceNames() + } + doc := logsDocument{header: newHeader(kindLogs), Stack: st.ref(a.shownPort(ctx))} + for _, name := range names { + if _, ok := p.Services[name]; !ok { + return fail.New(fail.InvalidArguments, "the stack has no service %q", name). + WithHint("its services are %v", p.ServiceNames()) + } + sl := serviceLogs{Name: name} + for _, c := range grouped[name] { + lines, err := a.opts.Docker.Logs(ctx, c.ID, tail) + if err != nil { + return err + } + sl.Lines = append(sl.Lines, r.Lines(lines)...) + } + doc.Services = append(doc.Services, sl) + } + slices.SortFunc(doc.Services, func(x, y serviceLogs) int { return strings.Compare(x.Name, y.Name) }) + return printJSON(cmd.OutOrStdout(), doc) + }, + } + cmd.Flags().BoolVarP(&follow, "follow", "f", false, "keep printing new lines") + cmd.Flags().IntVar(&tail, "tail", 0, "only the last n lines of each service") + cmd.Flags().StringVar(&file, "file", "", "compose file (default: compose.yaml or another name Compose looks for)") + registerJSON(cmd, &asJSON) + return cmd +} diff --git a/internal/cli/logs_test.go b/internal/cli/logs_test.go new file mode 100644 index 0000000..c17b652 --- /dev/null +++ b/internal/cli/logs_test.go @@ -0,0 +1,62 @@ +package cli + +import ( + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/testutil" +) + +func TestLogs(t *testing.T) { + t.Parallel() + + t.Run("json, redacted", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "db", "mail", "web") + e.docker.logs["shop-main-api-1"] = []string{"one", "token not-a-real-api-token accepted", "three"} + e.docker.logs["shop-main-db-1"] = []string{"listening on 5432", "password not-a-real-token"} + code, stdout, _ := e.run("logs", "--json", "--tail", "2", "api", "db") + if code != exitOK { + t.Fatalf("exit %d: %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "logs.json"), []byte(stdout)) + if strings.Contains(stdout, "not-a-real") { + t.Error("a secret reached the document") + } + }) + + t.Run("text goes through compose", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.stackUp(e.id(), "api") + if code, _, stderr := e.run("logs", "-f", "--tail", "5", "api"); code != exitOK { + t.Fatalf("exit %d: %s", code, stderr) + } + runs := e.compose.recorded() + if len(runs) != 1 || runs[0].inv.Project != "shop-main" || len(runs[0].inv.Files) != 0 || + !slices.Equal(runs[0].args, []string{"logs", "--follow", "--tail", "5", "api"}) { + t.Errorf("compose runs %+v", runs) + } + }) + + t.Run("json and follow", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + if code, stdout, _ := e.run("logs", "--json", "-f"); code != exitFailed || !strings.Contains(stdout, `"code": "InvalidArguments"`) { + t.Errorf("exit %d, stdout %s", code, stdout) + } + }) + + t.Run("unknown service", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.stackUp(e.id(), "api") + if code, stdout, _ := e.run("logs", "--json", "nope"); code != exitFailed || !strings.Contains(stdout, `"code": "InvalidArguments"`) { + t.Errorf("exit %d, stdout %s", code, stdout) + } + }) +} diff --git a/internal/cli/open.go b/internal/cli/open.go new file mode 100644 index 0000000..d778b5a --- /dev/null +++ b/internal/cli/open.go @@ -0,0 +1,85 @@ +package cli + +import ( + "fmt" + "os/exec" + "runtime" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" +) + +func newOpenCommand(a *app) *cobra.Command { + var asJSON, printOnly bool + var file string + cmd := &cobra.Command{ + Use: "open [service]", + Short: "Open the stack's address, or a service's, in the browser", + Args: cobra.MaximumNArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + ctx := cmd.Context() + st, err := a.findStack(ctx) + if err != nil { + return err + } + port := a.shownPort(ctx) + doc := addressDocument{header: newHeader(kindAddress), URL: address(st.id.Host(), port, "/")} + if len(args) == 1 { + _, p, err := a.loadProject(ctx, st.wt.Top, file, st.project) + if err != nil { + return err + } + routes, _, err := stack.Routes(p) + if err != nil { + return err + } + if _, ok := stack.Find(routes, args[0]); !ok { + names := make([]string, len(routes)) + for i, r := range routes { + names[i] = r.Service + } + return fail.New(fail.InvalidArguments, "the stack has no HTTP service %q", args[0]). + WithHint("its HTTP services are: %s", strings.Join(names, ", ")) + } + doc.Service = args[0] + doc.URL = address(st.id.ServiceHost(args[0]), port, "/") + } + switch { + case asJSON: + return printJSON(cmd.OutOrStdout(), doc) + case printOnly: + _, err := fmt.Fprintln(cmd.OutOrStdout(), doc.URL) + return err + } + if err := a.opts.Browser.Open(doc.URL); err != nil { + return fail.New(fail.Internal, "open %s: %v", doc.URL, err).WithHint("open it yourself, or print it with --print") + } + _, err = fmt.Fprintln(cmd.OutOrStdout(), doc.URL) + return err + }, + } + cmd.Flags().BoolVar(&printOnly, "print", false, "print the address instead of opening it") + cmd.Flags().StringVarP(&file, "file", "f", "", "compose file (default: compose.yaml or another name Compose looks for)") + registerJSON(cmd, &asJSON) + return cmd +} + +// SystemBrowser opens URLs with the platform's opener: open on macOS, $BROWSER (set by VS Code +// in a devcontainer) or xdg-open on Linux. +type SystemBrowser struct { + Getenv func(string) string +} + +// Open opens url. +func (b SystemBrowser) Open(url string) error { + if runtime.GOOS == "darwin" { + return exec.Command("open", url).Run() + } + if browser := b.Getenv("BROWSER"); browser != "" { + return exec.Command(browser, url).Run() + } + return exec.Command("xdg-open", url).Run() +} diff --git a/internal/cli/open_test.go b/internal/cli/open_test.go new file mode 100644 index 0000000..be4f890 --- /dev/null +++ b/internal/cli/open_test.go @@ -0,0 +1,68 @@ +package cli + +import ( + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/testutil" +) + +func TestOpen(t *testing.T) { + t.Parallel() + + for _, tt := range []struct { + name string + port int + args []string + want string + }{ + {name: "port 80", port: 80, args: []string{"open", "--print"}, want: "http://shop.main.localhost/\n"}, + {name: "port 8080", port: 8080, args: []string{"open", "--print"}, want: "http://shop.main.localhost:8080/\n"}, + {name: "a service", port: 8080, args: []string{"open", "--print", "mail"}, want: "http://mail.shop.main.localhost:8080/\n"}, + } { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(tt.port) + e.stackUp(e.id(), "web") + if code, stdout, _ := e.run(tt.args...); code != exitOK || stdout != tt.want || len(e.browser.opened) != 0 { + t.Errorf("exit %d, stdout %q, opened %v; want %q and no browser", code, stdout, e.browser.opened, tt.want) + } + }) + } + + t.Run("opens the browser", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "web") + if code, _, _ := e.run("open"); code != exitOK || !slices.Equal(e.browser.opened, []string{"http://shop.main.localhost/"}) { + t.Errorf("exit %d, opened %v", code, e.browser.opened) + } + }) + + t.Run("json", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "web") + code, stdout, _ := e.run("open", "api", "--json") + if code != exitOK || len(e.browser.opened) != 0 { + t.Fatalf("exit %d, opened %v", code, e.browser.opened) + } + testutil.Golden(t, filepath.Join("testdata", "json", "address.json"), []byte(stdout)) + }) + + t.Run("unknown service", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "web") + code, _, stderr := e.run("open", "nope") + if code != exitFailed || !strings.Contains(stderr, `no HTTP service "nope"`) || !strings.Contains(stderr, "api, mail, web") { + t.Errorf("exit %d, stderr %q", code, stderr) + } + }) +} diff --git a/internal/cli/ps.go b/internal/cli/ps.go new file mode 100644 index 0000000..47dc6b8 --- /dev/null +++ b/internal/cli/ps.go @@ -0,0 +1,144 @@ +package cli + +import ( + "context" + "errors" + "fmt" + "io/fs" + "os" + "sort" + "sync" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/stack" +) + +func newPsCommand(a *app) *cobra.Command { + var asJSON bool + cmd := &cobra.Command{ + Use: "ps", + Short: "List every stack on this machine with its state, address and memory", + Long: `List every stack on this machine, from any directory: repository, branch, state +(running, partial or stopped), address, memory in use and worktree. ps reads only Docker; the +full verdict on one stack is siding status.`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + doc, err := a.ps(cmd.Context()) + if err != nil { + return err + } + if asJSON { + return printJSON(cmd.OutOrStdout(), doc) + } + return printPsText(cmd, doc) + }, + } + registerJSON(cmd, &asJSON) + return cmd +} + +func (a *app) ps(ctx context.Context) (stackListDocument, error) { + doc := stackListDocument{header: newHeader(kindStackList)} + if err := a.opts.Docker.Ping(ctx); err != nil { + return doc, err + } + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelRepo: ""}) + if err != nil { + return doc, err + } + port := a.shownPort(ctx) + projects := map[string][]docker.Container{} + for _, c := range cs { + p := c.Labels[stack.LabelProject] + projects[p] = append(projects[p], c) + } + memory, err := a.memory(ctx, cs) + if err != nil { + return doc, err + } + for project, containers := range projects { + id := stack.FromLabels(containers[0].Labels) + e := stackEntry{Project: project, Repo: id.Repo, Branch: id.Branch, Worktree: id.Worktree, + Machine: id.Machine, Agent: id.Agent, URL: address(id.Host(), port, "/")} + running := 0 + for _, c := range containers { + if c.State == "running" { + running++ + } + e.MemoryBytes += memory[c.ID] + } + switch running { + case len(containers): + e.State = "running" + case 0: + e.State = "stopped" + default: + e.State = "partial" + } + if _, err := os.Stat(id.Worktree); errors.Is(err, fs.ErrNotExist) { + e.WorktreeMissing = true + } + doc.MemoryBytes += e.MemoryBytes + doc.Stacks = append(doc.Stacks, e) + } + sort.Slice(doc.Stacks, func(i, j int) bool { return doc.Stacks[i].Project < doc.Stacks[j].Project }) + return doc, nil +} + +// memory asks for the memory of every running container at once; a stats call takes a moment. +func (a *app) memory(ctx context.Context, cs []docker.Container) (map[string]uint64, error) { + var mu sync.Mutex + var wg sync.WaitGroup + out := map[string]uint64{} + var firstErr error + for _, c := range cs { + if c.State != "running" { + continue + } + wg.Add(1) + go func(id string) { + defer wg.Done() + m, err := a.opts.Docker.Memory(ctx, id) + mu.Lock() + defer mu.Unlock() + if err != nil { + if firstErr == nil { + firstErr = err + } + return + } + out[id] = m + }(c.ID) + } + wg.Wait() + return out, firstErr +} + +func printPsText(cmd *cobra.Command, doc stackListDocument) error { + if len(doc.Stacks) == 0 { + _, err := fmt.Fprintln(cmd.OutOrStdout(), "No stacks. Start one with siding up in a worktree.") + return err + } + rows := [][]string{{"REPO", "BRANCH", "STATE", "URL", "MEMORY", "WORKTREE"}} + for _, s := range doc.Stacks { + worktree := s.Worktree + if s.WorktreeMissing { + worktree += " (missing)" + } + rows = append(rows, []string{s.Repo, s.Branch, s.State, s.URL, bytesText(s.MemoryBytes), worktree}) + } + rows = append(rows, []string{"total", "", "", "", bytesText(doc.MemoryBytes), ""}) + _, err := fmt.Fprint(cmd.OutOrStdout(), table("", rows)) + return err +} + +// bytesText shows memory in MiB, or GiB from 1 GiB on. +func bytesText(n uint64) string { + const mib, gib = 1 << 20, 1 << 30 + if n >= gib { + return fmt.Sprintf("%.1f GiB", float64(n)/gib) + } + return fmt.Sprintf("%d MiB", (n+mib/2)/mib) +} diff --git a/internal/cli/ps_test.go b/internal/cli/ps_test.go new file mode 100644 index 0000000..aa2e10f --- /dev/null +++ b/internal/cli/ps_test.go @@ -0,0 +1,70 @@ +package cli + +import ( + "path/filepath" + "testing" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/testutil" +) + +func TestPs(t *testing.T) { + t.Parallel() + + t.Run("stacks", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(8080) + main := e.id() + login := e.id() + login.Branch, login.Worktree, login.Agent = "feature/login", filepath.Join(e.base, "shop-login"), "claude" + e.stackUp(main, "web", "db") + e.stackUp(login, "web", "db") + e.docker.update("db", func(c *docker.Container) { + if c.Labels["dev.siding.branch"] == "feature/login" { + c.State = "exited" + } + }) + e.docker.add(docker.Container{ID: "foreign", Labels: map[string]string{"com.docker.compose.project": "other"}, State: "running"}) + const mib = 1 << 20 + e.docker.memory["shop-main-web-1"] = 200 * mib + e.docker.memory["shop-main-db-1"] = 100 * mib + e.docker.memory["shop-feature-login-web-1"] = 500 * mib + + code, stdout, _ := e.run("ps", "--json") + if code != exitOK { + t.Fatalf("exit %d", code) + } + testutil.Golden(t, filepath.Join("testdata", "json", "stacklist.json"), []byte(stdout)) + + _, text, _ := e.run("ps") + want := `REPO BRANCH STATE URL MEMORY WORKTREE +shop feature/login partial http://shop.feature-login.localhost:8080/ 500 MiB /src/shop-login (missing) +shop main running http://shop.main.localhost:8080/ 300 MiB /src/shop +total 800 MiB +` + if text != want { + t.Errorf("text:\n%s\nwant:\n%s", text, want) + } + }) + + t.Run("no stacks", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + code, stdout, _ := e.run("ps", "--json") + if code != exitOK { + t.Fatalf("exit %d", code) + } + testutil.Golden(t, filepath.Join("testdata", "json", "stacklist-empty.json"), []byte(stdout)) + }) + + t.Run("from outside any repository", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.stackUp(e.id(), "web") + e.opts.Dir = t.TempDir() + if code, stdout, _ := e.run("ps"); code != exitOK || stdout == "" { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) +} diff --git a/internal/cli/setup.go b/internal/cli/setup.go new file mode 100644 index 0000000..6fc003e --- /dev/null +++ b/internal/cli/setup.go @@ -0,0 +1,120 @@ +package cli + +import ( + "errors" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/proxy" + "github.com/tweinmann/siding/internal/stack" +) + +func newSetupCommand(a *app) *cobra.Command { + var asJSON bool + var port int + cmd := &cobra.Command{ + Use: "setup", + Short: "Start the shared proxy siding-proxy on 127.0.0.1", + Long: `Start the shared proxy siding-proxy: Traefik on the network "siding", published on +127.0.0.1 only. Without --port it keeps the port of a running proxy, and uses 80 otherwise. +Running setup again changes nothing unless the port or the image differs.`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + if port < 0 || port > 65535 { + return fail.New(fail.InvalidArguments, "--port must be between 1 and 65535, got %d", port) + } + doc, err := a.setup(cmd, port) + if err != nil { + return err + } + if asJSON { + return printJSON(cmd.OutOrStdout(), doc) + } + verb := map[string]string{"created": "started", "updated": "restarted", "unchanged": "already runs"}[doc.State] + _, err = fmt.Fprintf(cmd.OutOrStdout(), "%s %s on %s\n", doc.Container, verb, address("*.localhost", doc.Port, "/")) + return err + }, + } + cmd.Flags().IntVar(&port, "port", 0, "port on 127.0.0.1 for the proxy (default: the running proxy's, else 80)") + registerJSON(cmd, &asJSON) + return cmd +} + +func (a *app) setup(cmd *cobra.Command, port int) (setupDocument, error) { + ctx := cmd.Context() + cfg := a.opts.Proxy + if err := a.opts.Docker.Ping(ctx); err != nil { + return setupDocument{}, err + } + machine, err := a.machineID() + if err != nil { + return setupDocument{}, err + } + current, err := a.opts.Docker.Container(ctx, cfg.Container) + if err != nil { + return setupDocument{}, err + } + currentPort := 0 + if current != nil { + for _, p := range current.Ports { + if p.ContainerPort == proxyContainerPort { + currentPort = p.HostPort + } + } + } + if port == 0 { + port = cfg.DefaultPort + if current != nil && current.State == "running" && currentPort != 0 { + port = currentPort + } + } + doc := setupDocument{header: newHeader(kindSetup), Container: cfg.Container, Network: cfg.Network, + Image: cfg.Image, Port: port, URL: address("*.localhost", port, "/")} + + if current != nil && current.State == "running" && currentPort == port && current.Image == cfg.Image { + doc.State = "unchanged" + return doc, nil + } + doc.State = "created" + if current != nil { + doc.State = "updated" + } + + if _, err := a.opts.Docker.EnsureNetwork(ctx, cfg.Network, map[string]string{stack.LabelMachine: machine}); err != nil { + return setupDocument{}, err + } + dir := filepath.Join(a.opts.Home, ".siding", "proxy", cfg.Project) + if err := os.MkdirAll(dir, 0o755); err != nil { + return setupDocument{}, fmt.Errorf("write the proxy's compose file: %w", err) + } + file := filepath.Join(dir, "compose.yml") + if err := os.WriteFile(file, proxy.ComposeFile(cfg, port, machine), 0o644); err != nil { + return setupDocument{}, fmt.Errorf("write the proxy's compose file: %w", err) + } + inv := compose.Invocation{Dir: dir, Project: cfg.Project, Files: []string{file}} + err = a.opts.Compose.Run(ctx, inv, []string{"up", "-d", "--wait", "--wait-timeout", "60", "--remove-orphans"}, cmd.ErrOrStderr(), cmd.ErrOrStderr()) + if err != nil { + return setupDocument{}, portInUse(err, port) + } + return doc, nil +} + +// portInUse turns Compose's "address already in use" into PortInUse. +func portInUse(err error, port int) error { + var e *fail.Error + if errors.As(err, &e) { + msg := strings.ToLower(e.Message) + if strings.Contains(msg, "address already in use") || strings.Contains(msg, "port is already allocated") || + strings.Contains(msg, "ports are not available") { + return fail.New(fail.PortInUse, "port %d on 127.0.0.1 is in use", port). + WithHint("free it, or choose another port with siding setup --port 8080") + } + } + return err +} diff --git a/internal/cli/setup_test.go b/internal/cli/setup_test.go new file mode 100644 index 0000000..0d7a1f3 --- /dev/null +++ b/internal/cli/setup_test.go @@ -0,0 +1,85 @@ +package cli + +import ( + "os" + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/testutil" +) + +func TestSetup(t *testing.T) { + t.Parallel() + + t.Run("first setup", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + code, stdout, stderr := e.run("setup", "--json") + if code != exitOK || stderr != "" { + t.Fatalf("exit %d, stderr %q", code, stderr) + } + testutil.Golden(t, filepath.Join("testdata", "json", "setup.json"), []byte(stdout)) + if !slices.Equal(e.docker.ensured, []string{"siding"}) { + t.Errorf("networks ensured: %v", e.docker.ensured) + } + runs := e.compose.recorded() + if len(runs) != 1 || runs[0].inv.Project != "siding" || runs[0].args[0] != "up" { + t.Fatalf("compose runs %+v", runs) + } + data, err := os.ReadFile(runs[0].inv.Files[0]) + if err != nil { + t.Fatal(err) + } + for _, want := range []string{`"127.0.0.1:80:80"`, `container_name: "siding-proxy"`, "LabelRegex(`dev.siding.repo`, `.+`)", + "Label(`traefik.docker.network`, `siding`)", "--providers.docker.exposedbydefault=false"} { + if !strings.Contains(string(data), want) { + t.Errorf("the proxy's compose file lacks %s:\n%s", want, data) + } + } + if !strings.HasPrefix(runs[0].inv.Files[0], e.opts.Home+"/.siding/proxy/") { + t.Errorf("compose file %s is not under ~/.siding", runs[0].inv.Files[0]) + } + }) + + t.Run("already running", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + code, stdout, _ := e.run("setup") + if code != exitOK || stdout != "siding-proxy already runs on http://*.localhost/\n" || len(e.compose.recorded()) != 0 { + t.Errorf("exit %d, stdout %q, compose runs %d", code, stdout, len(e.compose.recorded())) + } + }) + + t.Run("keeps the running port without --port", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(8080) + if code, stdout, _ := e.run("setup"); code != exitOK || !strings.Contains(stdout, "already runs on http://*.localhost:8080/") { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) + + t.Run("another port recreates the proxy", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + code, stdout, _ := e.run("setup", "--port", "8080") + if code != exitOK || stdout != "siding-proxy restarted on http://*.localhost:8080/\n" || len(e.compose.recorded()) != 1 { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) + + t.Run("port in use", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.compose.runErr = fail.New(fail.ComposeFailed, "docker compose up failed: Error response from daemon: failed to bind host port 127.0.0.1:80/tcp: address already in use") + code, _, stderr := e.run("setup") + if code != exitFailed || !strings.Contains(stderr, "port 80 on 127.0.0.1 is in use") || !strings.Contains(stderr, "--port") { + t.Errorf("exit %d, stderr %q", code, stderr) + } + }) +} diff --git a/internal/cli/skill.go b/internal/cli/skill.go new file mode 100644 index 0000000..1b8b8cf --- /dev/null +++ b/internal/cli/skill.go @@ -0,0 +1,36 @@ +package cli + +import ( + _ "embed" + "io" + + "github.com/spf13/cobra" +) + +// skillText is the Claude Code skill siding ships, kept in the binary so that text and binary +// never drift apart. +// +//go:embed skill/SKILL.md +var skillText string + +func newSkillCommand() *cobra.Command { + var asJSON bool + cmd := &cobra.Command{ + Use: "skill", + Short: "Print a Claude Code skill that teaches an agent to use siding", + Long: `Print a Claude Code skill (SKILL.md) that tells an agent to start its stack and wait +for the verdict. siding writes no file; install it with + + mkdir -p .claude/skills/siding && siding skill > .claude/skills/siding/SKILL.md`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + if asJSON { + return printJSON(cmd.OutOrStdout(), skillDocument{header: newHeader(kindSkill), Name: "siding", Text: skillText}) + } + _, err := io.WriteString(cmd.OutOrStdout(), skillText) + return err + }, + } + registerJSON(cmd, &asJSON) + return cmd +} diff --git a/internal/cli/skill/SKILL.md b/internal/cli/skill/SKILL.md new file mode 100644 index 0000000..2f6d9c6 --- /dev/null +++ b/internal/cli/skill/SKILL.md @@ -0,0 +1,51 @@ +--- +name: siding +description: Start this worktree's Docker Compose stack with siding and wait for the verdict, to check that a change to the app actually runs. Use after changing code, Dockerfiles, compose files or configuration that the running app depends on, and before reporting such work as done. +--- + +# Check that the app runs with siding + +siding runs this git worktree's compose file as its own stack, reachable through a local proxy +at `http://..localhost`, and tells you whether it actually runs. Every command +below prints one JSON document with `--json`. + +## After a change that affects the running app + +Run: + +```sh +siding up --wait --json +``` + +It starts or updates the stack and waits, three minutes at most, for one `Status` document. +Read its `verdict` and the exit code: + +- **0, `Ready`**: every service runs, every healthcheck is healthy and every routed service + answers through the proxy. Report the address in `stack.url` (and a service's `url` if it + matters) together with your result. +- **1, `Failed`**: `failure` names the `service`, the `reason` (`Exited`, `Unhealthy`, + `Restarting`, `Missing`), a `hint` and the last log lines of that service in `logs`. Fix the + cause in that service, then run `siding up --wait --json` again. +- **2, `TimedOut`**: the stack was not ready in time. `failure` names the first service that + was not ready: `NotAnswering` means it runs but does not answer through the proxy (often it + listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady` + means it is still starting. Read the hint and the logs; if it is only slow, run + `siding status --wait --json --timeout 5m`. + +An `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`, +`message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`. + +## More detail + +- `siding status --json`: the current verdict, without waiting (exit 2 while starting). +- `siding logs --json` or `siding logs --json --tail 200`: the last log lines per + service. Values from `.env` and the services' environment appear as `***`. +- `siding ps --json`: every stack on this machine, with state and memory. + +## Rules + +- Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file + or the service labels `dev.siding.route` and `dev.siding.port` instead. +- Leave stopping and removing stacks to the user; `siding down` only removes containers, and + removing data is the user's decision. +- Do not paste log lines into places outside this conversation. diff --git a/internal/cli/skill_test.go b/internal/cli/skill_test.go new file mode 100644 index 0000000..e94cd55 --- /dev/null +++ b/internal/cli/skill_test.go @@ -0,0 +1,45 @@ +package cli + +import ( + "encoding/json" + "path/filepath" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/testutil" +) + +func TestSkill(t *testing.T) { + t.Parallel() + e := newEnv(t) + code, text, _ := e.run("skill") + if code != exitOK { + t.Fatalf("exit %d", code) + } + testutil.Golden(t, filepath.Join("testdata", "skill", "SKILL.md"), []byte(text)) + + if !strings.HasPrefix(text, "---\nname: siding\ndescription: ") { + t.Errorf("not a skill named siding:\n%s", text[:min(len(text), 80)]) + } + for _, want := range []string{"siding up --wait --json", "siding status --json", "siding logs --json", "**0, `Ready`**", "**1, `Failed`**", "**2, `TimedOut`**"} { + if !strings.Contains(text, want) { + t.Errorf("the skill does not mention %q", want) + } + } + // Only a human removes data, forces anything, or exposes a stack. + for _, never := range []string{"--force", "--volumes", "share"} { + if strings.Contains(text, never) { + t.Errorf("the skill mentions %q", never) + } + } + + code, stdout, _ := e.run("skill", "--json") + if code != exitOK { + t.Fatalf("exit %d", code) + } + testutil.Golden(t, filepath.Join("testdata", "json", "skill.json"), []byte(stdout)) + var doc skillDocument + if err := json.Unmarshal([]byte(stdout), &doc); err != nil || doc.Text != text || doc.Name != "siding" { + t.Errorf("document %+v, err %v", doc, err) + } +} diff --git a/internal/cli/stack.go b/internal/cli/stack.go new file mode 100644 index 0000000..a43897e --- /dev/null +++ b/internal/cli/stack.go @@ -0,0 +1,171 @@ +package cli + +import ( + "context" + "errors" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + "strconv" + "strings" + "text/tabwriter" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/stack" +) + +// proxyContainerPort is the port Traefik listens on inside siding-proxy. +const proxyContainerPort = 80 + +// existing is the stack of the current worktree, found through its labels. +type existing struct { + wt git.Worktree + id stack.Identity + project string +} + +func (s existing) ref(port int) stackRef { + return stackRef{Project: s.project, Repo: s.id.Repo, Branch: s.id.Branch, Worktree: s.id.Worktree, URL: address(s.id.Host(), port, "/")} +} + +// findStack returns the stack of the worktree containing the current directory: the project +// whose containers carry this worktree and branch. Asking Docker rather than recomputing the +// name keeps --repo, given once to up, out of every later command. +func (a *app) findStack(ctx context.Context) (existing, error) { + wt, err := a.opts.Git.Worktree(ctx, a.opts.Dir) + if err != nil { + return existing{}, err + } + if err := a.opts.Docker.Ping(ctx); err != nil { + return existing{}, err + } + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelWorktree: wt.Top, stack.LabelBranch: wt.Branch}) + if err != nil { + return existing{}, err + } + if len(cs) == 0 { + return existing{}, fail.New(fail.StackNotFound, "the worktree %s has no stack for the branch %s", wt.Top, wt.Branch). + WithHint("start it with siding up") + } + return existing{wt: wt, id: stack.FromLabels(cs[0].Labels), project: cs[0].Labels[stack.LabelProject]}, nil +} + +// proxyPort returns the host port of the running proxy. +func (a *app) proxyPort(ctx context.Context) (int, error) { + c, err := a.opts.Docker.Container(ctx, a.opts.Proxy.Container) + if err != nil { + return 0, err + } + if c == nil || c.State != "running" { + return 0, fail.New(fail.ProxyNotRunning, "%s is not running", a.opts.Proxy.Container). + WithHint("start it with siding setup") + } + for _, p := range c.Ports { + if p.ContainerPort == proxyContainerPort { + return p.HostPort, nil + } + } + return 0, fail.New(fail.ProxyNotRunning, "%s publishes no port", a.opts.Proxy.Container). + WithHint("recreate it with siding setup --port ") +} + +// shownPort is the proxy's port for addresses in ps, which also works while the proxy is down. +func (a *app) shownPort(ctx context.Context) int { + if port, err := a.proxyPort(ctx); err == nil { + return port + } + return a.opts.Proxy.DefaultPort +} + +// loadProject reads the compose configuration of a worktree as project. +func (a *app) loadProject(ctx context.Context, top, file, project string) (string, *compose.Project, error) { + path, err := compose.FindFile(a.opts.Dir, top, file) + if err != nil { + return "", nil, err + } + data, err := a.opts.Compose.Config(ctx, compose.Invocation{Dir: top, Project: project, Files: []string{path}}) + if err != nil { + return "", nil, err + } + p, err := compose.ParseConfig(data) + if err != nil { + return "", nil, fail.New(fail.ComposeFailed, "%v", err) + } + return path, p, nil +} + +// redactor redacts the values of the worktree's .env and of every service's environment. +func redactor(top string, p *compose.Project) (*stack.Redactor, error) { + var values []string + data, err := os.ReadFile(filepath.Join(top, ".env")) + switch { + case err == nil: + values = stack.DotEnvValues(string(data)) + case !errors.Is(err, fs.ErrNotExist): + return nil, fmt.Errorf("read .env: %w", err) + } + if p != nil { + for _, s := range p.Services { + for _, v := range s.Environment { + values = append(values, v) + } + } + } + return stack.NewRedactor(values...), nil +} + +// machineID returns dev.siding.machine for this machine. +func (a *app) machineID() (string, error) { + osID := "" + if a.opts.OSMachineID != nil { + osID = a.opts.OSMachineID() + } + return stack.MachineID(osID, a.opts.Home) +} + +// address is http://[:port]; port 80 is left out. +func address(host string, port int, path string) string { + if port == 80 || port == 0 { + return "http://" + host + path + } + return "http://" + host + ":" + strconv.Itoa(port) + path +} + +// byService groups containers by their Compose service. +func byService(cs []docker.Container) map[string][]docker.Container { + m := map[string][]docker.Container{} + for _, c := range cs { + s := c.Labels[stack.LabelService] + m[s] = append(m[s], c) + } + return m +} + +// table aligns rows into columns two spaces apart, each line starting with indent. +func table(indent string, rows [][]string) string { + var b strings.Builder + tw := tabwriter.NewWriter(&b, 0, 4, 2, ' ', 0) + for _, r := range rows { + _, _ = io.WriteString(tw, indent+strings.Join(r, "\t")+"\n") + } + _ = tw.Flush() + if b.Len() == 0 { + return "" + } + // tabwriter pads empty last columns with spaces. + lines := strings.Split(strings.TrimSuffix(b.String(), "\n"), "\n") + for i := range lines { + lines[i] = strings.TrimRight(lines[i], " ") + } + return strings.Join(lines, "\n") + "\n" +} + +// warnf prints a warning, on stderr. +func warnf(w io.Writer, format string, args ...any) { + _, _ = fmt.Fprintf(w, "warning: "+format+"\n", args...) +} diff --git a/internal/cli/status.go b/internal/cli/status.go new file mode 100644 index 0000000..88d822f --- /dev/null +++ b/internal/cli/status.go @@ -0,0 +1,229 @@ +package cli + +import ( + "context" + "fmt" + "io" + "strconv" + "strings" + "time" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/stack" + "github.com/tweinmann/siding/internal/verdict" +) + +const ( + // defaultTimeout is how long --wait waits by default. + defaultTimeout = 3 * time.Minute + // pollInterval is the time between two checks while waiting. + pollInterval = time.Second + // logTail is how many log lines a failure carries. + logTail = 20 +) + +type waitFlags struct { + wait bool + timeout time.Duration +} + +func (w *waitFlags) register(cmd *cobra.Command) { + cmd.Flags().BoolVar(&w.wait, "wait", false, "wait until the stack is Ready or Failed, at most --timeout") + cmd.Flags().DurationVar(&w.timeout, "timeout", defaultTimeout, "how long --wait waits") +} + +func newStatusCommand(a *app) *cobra.Command { + var asJSON bool + var file string + var w waitFlags + cmd := &cobra.Command{ + Use: "status", + Short: "Tell whether the worktree's stack runs: Ready, Starting, Failed or TimedOut", + Long: `Tell whether the worktree's stack runs. Ready means every service has a container, +every healthcheck is healthy, and every service with a route path answers through the proxy. +A failure names the service, the reason, a hint and its last log lines. + +Exit codes: 0 Ready, 1 Failed, 2 TimedOut (or Starting without --wait).`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + ctx := cmd.Context() + st, err := a.findStack(ctx) + if err != nil { + return err + } + port, err := a.proxyPort(ctx) + if err != nil { + return err + } + _, p, err := a.loadProject(ctx, st.wt.Top, file, st.project) + if err != nil { + return err + } + doc, err := a.status(ctx, st, p, port, w) + if err != nil { + return err + } + return printStatus(cmd.OutOrStdout(), doc, asJSON) + }, + } + cmd.Flags().StringVarP(&file, "file", "f", "", "compose file (default: compose.yaml or another name Compose looks for)") + w.register(cmd) + registerJSON(cmd, &asJSON) + return cmd +} + +// status evaluates the stack once, or until it is Ready or Failed or the time is up. +func (a *app) status(ctx context.Context, st existing, p *compose.Project, port int, w waitFlags) (statusDocument, error) { + routes, _, err := stack.Routes(p) + if err != nil { + return statusDocument{}, err + } + routed := map[string]verdict.Route{} + for _, r := range routes { + if r.Path != "" { + routed[r.Service] = verdict.Route{URL: address(st.id.Host(), port, r.Path), Port: r.Port} + } + } + start := a.opts.Clock.Now() + var base map[string]int + for { + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelProject: st.project}) + if err != nil { + return statusDocument{}, err + } + if base == nil { + base = map[string]int{} + for _, c := range cs { + base[c.ID] = c.RestartCount + } + } + answering := a.probe(ctx, st, routes, cs, port) + final := w.wait && a.opts.Clock.Now().Sub(start) >= w.timeout + res := verdict.Evaluate(verdict.Input{ + Services: p.ServiceNames(), Containers: cs, Routes: routed, Answering: answering, + RestartBase: base, Final: final, Timeout: w.timeout, + }) + if !w.wait || final || res.Verdict == verdict.Ready || res.Verdict == verdict.Failed { + return a.statusDocument(ctx, st, p, routes, port, cs, res) + } + if err := a.opts.Clock.Sleep(ctx, pollInterval); err != nil { + return statusDocument{}, err + } + } +} + +// probe asks the proxy, for every running service with a route path, whether it answers. +func (a *app) probe(ctx context.Context, st existing, routes []stack.Route, cs []docker.Container, port int) map[string]bool { + running := map[string]bool{} + for _, c := range cs { + if c.State == "running" && c.Health != "starting" && c.Health != "unhealthy" { + running[c.Labels[stack.LabelService]] = true + } + } + answering := map[string]bool{} + for _, r := range routes { + if r.Path == "" || !running[r.Service] { + continue + } + resp, err := a.opts.Probe.Get(ctx, "127.0.0.1:"+strconv.Itoa(port), st.id.Host(), r.Path) + answering[r.Service] = err == nil && resp.Answered(st.project+"/"+r.Service) + } + return answering +} + +func (a *app) statusDocument(ctx context.Context, st existing, p *compose.Project, routes []stack.Route, port int, + cs []docker.Container, res verdict.Result) (statusDocument, error) { + doc := statusDocument{header: newHeader(kindStatus), Verdict: res.Verdict, Stack: st.ref(port)} + for _, s := range res.Services { + ss := serviceStatus{Name: s.Name, State: s.State, Health: s.Health, Answering: s.Answering, Ready: s.Ready} + if r, ok := stack.Find(routes, s.Name); ok { + ss.Route = r.Path + ss.URL = address(st.id.ServiceHost(s.Name), port, "/") + } + doc.Services = append(doc.Services, ss) + } + if f := res.Failure; f != nil { + doc.Failure = &failureDocument{Service: f.Service, Reason: f.Reason, Hint: f.Hint} + if containers := byService(cs)[f.Service]; len(containers) > 0 { + lines, err := a.opts.Docker.Logs(ctx, failing(containers).ID, logTail) + if err != nil { + return statusDocument{}, err + } + r, err := redactor(st.wt.Top, p) + if err != nil { + return statusDocument{}, err + } + doc.Failure.Logs = r.Lines(lines) + } + } + return doc, nil +} + +// failing picks the container of a service whose logs explain its failure: one that is not +// running, else the first. +func failing(cs []docker.Container) docker.Container { + for _, c := range cs { + if c.State != "running" || c.Health == "unhealthy" { + return c + } + } + return cs[0] +} + +// printStatus prints the verdict and returns its exit code. +func printStatus(w io.Writer, doc statusDocument, asJSON bool) error { + if asJSON { + if err := printJSON(w, doc); err != nil { + return err + } + } else if err := printStatusText(w, doc); err != nil { + return err + } + switch doc.Verdict { + case verdict.Ready: + return nil + case verdict.Failed: + return exitCode(exitFailed) + default: + return exitCode(exitTimedOut) + } +} + +func printStatusText(w io.Writer, doc statusDocument) error { + var b strings.Builder + fmt.Fprintf(&b, "%s: %s %s\n", doc.Stack.Project, doc.Verdict, doc.Stack.URL) + var rows [][]string + for _, s := range doc.Services { + state := s.State + if s.Health != "" { + state += " (" + s.Health + ")" + } + reach := "" + if s.Answering != nil { + reach = "not answering" + if *s.Answering { + reach = "answering" + } + } + where := s.URL + if s.Route != "" { + where += " route " + s.Route + } + rows = append(rows, []string{s.Name, state, reach, where}) + } + b.WriteString(table(" ", rows)) + if f := doc.Failure; f != nil { + fmt.Fprintf(&b, "\n%s: %s, %s\n", f.Service, f.Reason, f.Hint) + if len(f.Logs) > 0 { + fmt.Fprintf(&b, "last log lines of %s:\n", f.Service) + for _, l := range f.Logs { + fmt.Fprintf(&b, " | %s\n", l) + } + } + } + _, err := io.WriteString(w, b.String()) + return err +} diff --git a/internal/cli/status_test.go b/internal/cli/status_test.go new file mode 100644 index 0000000..f6a357e --- /dev/null +++ b/internal/cli/status_test.go @@ -0,0 +1,175 @@ +package cli + +import ( + "encoding/json" + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/testutil" +) + +func TestStatus(t *testing.T) { + t.Parallel() + + t.Run("ready", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "mail", "web") + e.docker.add(container(e.id(), "db", "running", "healthy", 0)) + code, stdout, stderr := e.run("status", "--json") + if code != exitOK || stderr != "" { + t.Fatalf("exit %d, stderr %q, stdout %s", code, stderr, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "status-ready.json"), []byte(stdout)) + if !slices.Contains(e.probe.asked, "127.0.0.1:80 shop.main.localhost/api") { + t.Errorf("probes %v", e.probe.asked) + } + }) + + t.Run("failed with the log tail, redacted", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "db", "mail", "web") + api := container(e.id(), "api", "exited", "", 1) + e.docker.add(api) + e.docker.logs[api.ID] = []string{"starting", "connecting to postgres://app:not-a-real-token@db/app", "cannot connect to db"} + code, stdout, _ := e.run("status", "--json") + if code != exitFailed { + t.Fatalf("exit %d, stdout %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "status-failed.json"), []byte(stdout)) + if strings.Contains(stdout, "not-a-real-token") { + t.Error("a .env value reached the document") + } + }) + + t.Run("text of a failure", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "mail", "web") + db := container(e.id(), "db", "running", "unhealthy", 0) + db.HealthOutput = "pg_isready: no response" + e.docker.add(db) + e.docker.logs[db.ID] = []string{"FATAL: password authentication failed"} + code, stdout, _ := e.run("status") + want := `shop-main: Failed http://shop.main.localhost/ + api running answering http://api.shop.main.localhost/ route /api + db running (unhealthy) + mail running http://mail.shop.main.localhost/ + web running answering http://web.shop.main.localhost/ route / + +db: Unhealthy, its healthcheck fails: pg_isready: no response +last log lines of db: + | FATAL: password authentication failed +` + if code != exitFailed || stdout != want { + t.Errorf("exit %d, stdout:\n%s\nwant:\n%s", code, stdout, want) + } + }) + + t.Run("timed out", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "db", "mail", "web") + delete(e.probe.answers, "/") // web runs but never answers + code, stdout, _ := e.run("status", "--wait", "--timeout", "10s", "--json") + if code != exitTimedOut { + t.Fatalf("exit %d, stdout %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "status-timedout.json"), []byte(stdout)) + if e.clock.slept != 10 { + t.Errorf("slept %d times, want 10 one-second polls", e.clock.slept) + } + }) + + t.Run("waits until ready", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "mail", "web") + e.docker.add(container(e.id(), "db", "running", "starting", 0)) + e.clock.onSleep = func(n int) { + if n == 3 { + e.docker.update("db", func(c *docker.Container) { c.Health = "healthy" }) + } + } + code, stdout, _ := e.run("status", "--wait", "--json") + if code != exitOK || verdictOf(t, stdout) != "Ready" || e.clock.slept != 3 { + t.Errorf("exit %d, verdict %s, slept %d", code, verdictOf(t, stdout), e.clock.slept) + } + }) + + t.Run("starting without wait", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.stackUp(e.id(), "api", "mail", "web") + e.docker.add(container(e.id(), "db", "running", "starting", 0)) + code, stdout, _ := e.run("status", "--json") + if code != exitTimedOut || verdictOf(t, stdout) != "Starting" || e.clock.slept != 0 { + t.Errorf("exit %d, verdict %s", code, verdictOf(t, stdout)) + } + }) + + for _, tt := range []struct { + name string + setup func(e *env) + code string + }{ + {name: "no stack", setup: func(e *env) { e.proxyUp(80) }, code: fail.StackNotFound}, + {name: "no proxy", setup: func(e *env) { e.stackUp(e.id(), "web") }, code: fail.ProxyNotRunning}, + {name: "another branch's stack", setup: func(e *env) { + e.proxyUp(80) + other := e.id() + other.Branch = "feature/login" + e.stackUp(other, "web") + }, code: fail.StackNotFound}, + } { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + e := newEnv(t) + tt.setup(e) + if code, stdout, _ := e.run("status", "--json"); code != exitFailed || !strings.Contains(stdout, `"code": "`+tt.code+`"`) { + t.Errorf("exit %d, stdout %s", code, stdout) + } + }) + } +} + +func TestUpWait(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.compose.onRun = func(composeRun) { + e.stackUp(e.id(), "api", "db", "mail", "web") + } + code, stdout, stderr := e.run("up", "--wait", "--json") + if code != exitOK || verdictOf(t, stdout) != "Ready" || stderr != "" { + t.Errorf("exit %d, stdout %s, stderr %q", code, stdout, stderr) + } + + e.docker.update("api", func(c *docker.Container) { c.State, c.ExitCode = "exited", 1 }) + if code, stdout, _ := e.run("up", "--wait", "--json"); code != exitFailed || verdictOf(t, stdout) != "Failed" { + t.Errorf("after a crash: exit %d, verdict %s", code, verdictOf(t, stdout)) + } +} + +// verdictOf reads the verdict of one Status document, and fails the test unless stdout is +// exactly one document. +func verdictOf(t *testing.T, stdout string) string { + t.Helper() + var doc statusDocument + dec := json.NewDecoder(strings.NewReader(stdout)) + if err := dec.Decode(&doc); err != nil || dec.More() || doc.Kind != kindStatus { + t.Fatalf("not one Status document (%v): %s", err, stdout) + } + return doc.Verdict +} diff --git a/internal/cli/testdata/json/address.json b/internal/cli/testdata/json/address.json new file mode 100644 index 0000000..1bc1782 --- /dev/null +++ b/internal/cli/testdata/json/address.json @@ -0,0 +1,6 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Address", + "url": "http://api.shop.main.localhost/", + "service": "api" +} diff --git a/internal/cli/testdata/json/down-dryrun.json b/internal/cli/testdata/json/down-dryrun.json new file mode 100644 index 0000000..b775c4f --- /dev/null +++ b/internal/cli/testdata/json/down-dryrun.json @@ -0,0 +1,22 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Down", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "dryRun": true, + "containers": [ + "shop-main-db-1", + "shop-main-web-1" + ], + "networks": [ + "shop-main_default" + ], + "volumes": [ + "shop-main_data" + ] +} diff --git a/internal/cli/testdata/json/down.json b/internal/cli/testdata/json/down.json new file mode 100644 index 0000000..f67bdf8 --- /dev/null +++ b/internal/cli/testdata/json/down.json @@ -0,0 +1,18 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Down", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "containers": [ + "shop-main-db-1", + "shop-main-web-1" + ], + "networks": [ + "shop-main_default" + ] +} diff --git a/internal/cli/testdata/json/logs.json b/internal/cli/testdata/json/logs.json new file mode 100644 index 0000000..5d457cb --- /dev/null +++ b/internal/cli/testdata/json/logs.json @@ -0,0 +1,27 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Logs", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "lines": [ + "token *** accepted", + "three" + ] + }, + { + "name": "db", + "lines": [ + "listening on 5432", + "password ***" + ] + } + ] +} diff --git a/internal/cli/testdata/json/setup.json b/internal/cli/testdata/json/setup.json new file mode 100644 index 0000000..9c0f34c --- /dev/null +++ b/internal/cli/testdata/json/setup.json @@ -0,0 +1,10 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Setup", + "container": "siding-proxy", + "network": "siding", + "image": "traefik:v3.7.16", + "port": 80, + "url": "http://*.localhost/", + "state": "created" +} diff --git a/internal/cli/testdata/json/skill.json b/internal/cli/testdata/json/skill.json new file mode 100644 index 0000000..e5a492d --- /dev/null +++ b/internal/cli/testdata/json/skill.json @@ -0,0 +1,6 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Skill", + "name": "siding", + "text": "---\nname: siding\ndescription: Start this worktree's Docker Compose stack with siding and wait for the verdict, to check that a change to the app actually runs. Use after changing code, Dockerfiles, compose files or configuration that the running app depends on, and before reporting such work as done.\n---\n\n# Check that the app runs with siding\n\nsiding runs this git worktree's compose file as its own stack, reachable through a local proxy\nat `http://..localhost`, and tells you whether it actually runs. Every command\nbelow prints one JSON document with `--json`.\n\n## After a change that affects the running app\n\nRun:\n\n```sh\nsiding up --wait --json\n```\n\nIt starts or updates the stack and waits, three minutes at most, for one `Status` document.\nRead its `verdict` and the exit code:\n\n- **0, `Ready`**: every service runs, every healthcheck is healthy and every routed service\n answers through the proxy. Report the address in `stack.url` (and a service's `url` if it\n matters) together with your result.\n- **1, `Failed`**: `failure` names the `service`, the `reason` (`Exited`, `Unhealthy`,\n `Restarting`, `Missing`), a `hint` and the last log lines of that service in `logs`. Fix the\n cause in that service, then run `siding up --wait --json` again.\n- **2, `TimedOut`**: the stack was not ready in time. `failure` names the first service that\n was not ready: `NotAnswering` means it runs but does not answer through the proxy (often it\n listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady`\n means it is still starting. Read the hint and the logs; if it is only slow, run\n `siding status --wait --json --timeout 5m`.\n\nAn `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`,\n`message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`.\n\n## More detail\n\n- `siding status --json`: the current verdict, without waiting (exit 2 while starting).\n- `siding logs --json` or `siding logs --json --tail 200`: the last log lines per\n service. Values from `.env` and the services' environment appear as `***`.\n- `siding ps --json`: every stack on this machine, with state and memory.\n\n## Rules\n\n- Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file\n or the service labels `dev.siding.route` and `dev.siding.port` instead.\n- Leave stopping and removing stacks to the user; `siding down` only removes containers, and\n removing data is the user's decision.\n- Do not paste log lines into places outside this conversation.\n" +} diff --git a/internal/cli/testdata/json/stack.json b/internal/cli/testdata/json/stack.json new file mode 100644 index 0000000..33ac795 --- /dev/null +++ b/internal/cli/testdata/json/stack.json @@ -0,0 +1,30 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Stack", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "url": "http://api.shop.main.localhost/", + "route": "/api", + "port": 8080 + }, + { + "name": "mail", + "url": "http://mail.shop.main.localhost/", + "port": 8025 + }, + { + "name": "web", + "url": "http://web.shop.main.localhost/", + "route": "/", + "port": 3000 + } + ] +} diff --git a/internal/cli/testdata/json/stacklist-empty.json b/internal/cli/testdata/json/stacklist-empty.json new file mode 100644 index 0000000..e5eea1c --- /dev/null +++ b/internal/cli/testdata/json/stacklist-empty.json @@ -0,0 +1,4 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "StackList" +} diff --git a/internal/cli/testdata/json/stacklist.json b/internal/cli/testdata/json/stacklist.json new file mode 100644 index 0000000..8028677 --- /dev/null +++ b/internal/cli/testdata/json/stacklist.json @@ -0,0 +1,29 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "StackList", + "stacks": [ + { + "project": "shop-feature-login", + "repo": "shop", + "branch": "feature/login", + "worktree": "/src/shop-login", + "machine": "5c833a2dc2c206c2", + "agent": "claude", + "state": "partial", + "url": "http://shop.feature-login.localhost:8080/", + "memoryBytes": 524288000, + "worktreeMissing": true + }, + { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "machine": "5c833a2dc2c206c2", + "state": "running", + "url": "http://shop.main.localhost:8080/", + "memoryBytes": 314572800 + } + ], + "memoryBytes": 838860800 +} diff --git a/internal/cli/testdata/json/status-failed.json b/internal/cli/testdata/json/status-failed.json new file mode 100644 index 0000000..eef8687 --- /dev/null +++ b/internal/cli/testdata/json/status-failed.json @@ -0,0 +1,51 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Status", + "verdict": "Failed", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "state": "exited", + "route": "/api", + "url": "http://api.shop.main.localhost/", + "answering": false, + "ready": false + }, + { + "name": "db", + "state": "running", + "ready": true + }, + { + "name": "mail", + "state": "running", + "url": "http://mail.shop.main.localhost/", + "ready": true + }, + { + "name": "web", + "state": "running", + "route": "/", + "url": "http://web.shop.main.localhost/", + "answering": true, + "ready": true + } + ], + "failure": { + "service": "api", + "reason": "Exited", + "hint": "exited with code 1; see its last log lines", + "logs": [ + "starting", + "connecting to postgres://app:***@db/app", + "cannot connect to db" + ] + } +} diff --git a/internal/cli/testdata/json/status-ready.json b/internal/cli/testdata/json/status-ready.json new file mode 100644 index 0000000..cbe6652 --- /dev/null +++ b/internal/cli/testdata/json/status-ready.json @@ -0,0 +1,42 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Status", + "verdict": "Ready", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "state": "running", + "route": "/api", + "url": "http://api.shop.main.localhost/", + "answering": true, + "ready": true + }, + { + "name": "db", + "state": "running", + "health": "healthy", + "ready": true + }, + { + "name": "mail", + "state": "running", + "url": "http://mail.shop.main.localhost/", + "ready": true + }, + { + "name": "web", + "state": "running", + "route": "/", + "url": "http://web.shop.main.localhost/", + "answering": true, + "ready": true + } + ] +} diff --git a/internal/cli/testdata/json/status-timedout.json b/internal/cli/testdata/json/status-timedout.json new file mode 100644 index 0000000..aeb491c --- /dev/null +++ b/internal/cli/testdata/json/status-timedout.json @@ -0,0 +1,46 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Status", + "verdict": "TimedOut", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "state": "running", + "route": "/api", + "url": "http://api.shop.main.localhost/", + "answering": true, + "ready": true + }, + { + "name": "db", + "state": "running", + "ready": true + }, + { + "name": "mail", + "state": "running", + "url": "http://mail.shop.main.localhost/", + "ready": true + }, + { + "name": "web", + "state": "running", + "route": "/", + "url": "http://web.shop.main.localhost/", + "answering": false, + "ready": false + } + ], + "failure": { + "service": "web", + "reason": "NotAnswering", + "hint": "it runs, but http://shop.main.localhost/ does not answer through the proxy; check that it listens on port 3000 on 0.0.0.0, not only on 127.0.0.1" + } +} diff --git a/internal/cli/testdata/skill/SKILL.md b/internal/cli/testdata/skill/SKILL.md new file mode 100644 index 0000000..2f6d9c6 --- /dev/null +++ b/internal/cli/testdata/skill/SKILL.md @@ -0,0 +1,51 @@ +--- +name: siding +description: Start this worktree's Docker Compose stack with siding and wait for the verdict, to check that a change to the app actually runs. Use after changing code, Dockerfiles, compose files or configuration that the running app depends on, and before reporting such work as done. +--- + +# Check that the app runs with siding + +siding runs this git worktree's compose file as its own stack, reachable through a local proxy +at `http://..localhost`, and tells you whether it actually runs. Every command +below prints one JSON document with `--json`. + +## After a change that affects the running app + +Run: + +```sh +siding up --wait --json +``` + +It starts or updates the stack and waits, three minutes at most, for one `Status` document. +Read its `verdict` and the exit code: + +- **0, `Ready`**: every service runs, every healthcheck is healthy and every routed service + answers through the proxy. Report the address in `stack.url` (and a service's `url` if it + matters) together with your result. +- **1, `Failed`**: `failure` names the `service`, the `reason` (`Exited`, `Unhealthy`, + `Restarting`, `Missing`), a `hint` and the last log lines of that service in `logs`. Fix the + cause in that service, then run `siding up --wait --json` again. +- **2, `TimedOut`**: the stack was not ready in time. `failure` names the first service that + was not ready: `NotAnswering` means it runs but does not answer through the proxy (often it + listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady` + means it is still starting. Read the hint and the logs; if it is only slow, run + `siding status --wait --json --timeout 5m`. + +An `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`, +`message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`. + +## More detail + +- `siding status --json`: the current verdict, without waiting (exit 2 while starting). +- `siding logs --json` or `siding logs --json --tail 200`: the last log lines per + service. Values from `.env` and the services' environment appear as `***`. +- `siding ps --json`: every stack on this machine, with state and memory. + +## Rules + +- Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file + or the service labels `dev.siding.route` and `dev.siding.port` instead. +- Leave stopping and removing stacks to the user; `siding down` only removes containers, and + removing data is the user's decision. +- Do not paste log lines into places outside this conversation. diff --git a/internal/cli/up.go b/internal/cli/up.go new file mode 100644 index 0000000..0b4ac85 --- /dev/null +++ b/internal/cli/up.go @@ -0,0 +1,173 @@ +package cli + +import ( + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" +) + +func newUpCommand(a *app) *cobra.Command { + var asJSON bool + var f upFlags + var w waitFlags + cmd := &cobra.Command{ + Use: "up [-- ]", + Short: "Start the worktree's stack from its unchanged compose file", + Long: `Start the worktree's stack: the compose file as the Compose project -, +plus .siding/override.yml, which resets every published port and routes the HTTP services +through siding-proxy. The compose file is never edited. + +Arguments after -- go to docker compose up, e.g. siding up -- --build web. +With --wait, up ends like siding status --wait, with its verdict and exit code.`, + Args: func(cmd *cobra.Command, args []string) error { + if n := cmd.ArgsLenAtDash(); n > 0 || (n < 0 && len(args) > 0) { + return fail.New(fail.InvalidArguments, "siding up takes no arguments before --, got %q", args[0]). + WithHint("pass docker compose arguments after --, e.g. siding up -- --build") + } + return nil + }, + RunE: func(cmd *cobra.Command, args []string) error { + if n := cmd.ArgsLenAtDash(); n >= 0 { + f.compose = args[n:] + } + if f.agent == "" && a.opts.Getenv != nil { + f.agent = a.opts.Getenv("SIDING_AGENT") + } + return a.up(cmd, f, w, asJSON) + }, + } + cmd.Flags().StringVarP(&f.file, "file", "f", "", "compose file (default: compose.yaml or another name Compose looks for)") + cmd.Flags().StringVar(&f.repo, "repo", "", "repository name (default: the main worktree's directory name)") + cmd.Flags().StringVar(&f.agent, "agent", "", "agent name for the label dev.siding.agent (default: $SIDING_AGENT)") + w.register(cmd) + registerJSON(cmd, &asJSON) + return cmd +} + +type upFlags struct { + file, repo, agent string + compose []string +} + +func (a *app) up(cmd *cobra.Command, f upFlags, w waitFlags, asJSON bool) error { + ctx := cmd.Context() + stderr := cmd.ErrOrStderr() + wt, err := a.opts.Git.Worktree(ctx, a.opts.Dir) + if err != nil { + return err + } + machine, err := a.machineID() + if err != nil { + return err + } + repo := f.repo + if repo == "" { + repo = wt.RepoName() + } + id := stack.Identity{Repo: repo, Branch: wt.Branch, Worktree: wt.Top, Machine: machine, Agent: f.agent} + project := id.Project() + + if err := a.opts.Docker.Ping(ctx); err != nil { + return err + } + port, err := a.proxyPort(ctx) + if err != nil { + return err + } + existingContainers, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelProject: project}) + if err != nil { + return err + } + for _, c := range existingContainers { + other := stack.FromLabels(c.Labels) + // A plain Compose project without siding's labels is adopted. + if other.Worktree == "" { + continue + } + if other.Worktree != id.Worktree || other.Branch != id.Branch { + return fail.New(fail.SlugCollision, "the Compose project %s already belongs to the branch %s in %s", project, other.Branch, other.Worktree). + WithHint("the branches %q and %q have the same slug; rename one, or pass --repo to tell them apart", other.Branch, id.Branch) + } + } + + file, p, err := a.loadProject(ctx, wt.Top, f.file, project) + if err != nil { + return err + } + routes, warnings, err := stack.Routes(p) + if err != nil { + return err + } + for _, w := range warnings { + warnf(stderr, "%s", w) + } + override, err := writeOverride(wt.Top, stack.Override(p, id, routes, a.opts.Proxy.Network)) + if err != nil { + return err + } + + inv := compose.Invocation{Dir: wt.Top, Project: project, Files: []string{file, override}} + args := append([]string{"up", "-d", "--remove-orphans"}, f.compose...) + // Compose's progress, and anything it prints, goes to stderr: stdout is for siding's result. + if err := a.opts.Compose.Run(ctx, inv, args, stderr, stderr); err != nil { + return err + } + + st := existing{wt: wt, id: id, project: project} + if w.wait { + doc, err := a.status(ctx, st, p, port, w) + if err != nil { + return err + } + return printStatus(cmd.OutOrStdout(), doc, asJSON) + } + doc := stackDocument{header: newHeader(kindStack), Stack: st.ref(port)} + for _, r := range routes { + doc.Services = append(doc.Services, serviceAddress{Name: r.Service, URL: address(id.ServiceHost(r.Service), port, "/"), Route: r.Path, Port: r.Port}) + } + if asJSON { + return printJSON(cmd.OutOrStdout(), doc) + } + return printStackText(cmd, doc) +} + +// writeOverride writes .siding/override.yml and .siding/.gitignore, which ignores the whole +// directory, itself included, so .siding/ never makes the worktree dirty. +func writeOverride(top string, content []byte) (string, error) { + path := filepath.Join(top, stack.OverrideFile) + dir := filepath.Dir(path) + if err := os.MkdirAll(dir, 0o755); err != nil { + return "", fmt.Errorf("write %s: %w", stack.OverrideFile, err) + } + if err := os.WriteFile(filepath.Join(dir, ".gitignore"), []byte("# Written by siding: nothing in .siding/ belongs in git.\n*\n"), 0o644); err != nil { + return "", fmt.Errorf("write .siding/.gitignore: %w", err) + } + if err := os.WriteFile(path, content, 0o644); err != nil { + return "", fmt.Errorf("write %s: %w", stack.OverrideFile, err) + } + return path, nil +} + +func printStackText(cmd *cobra.Command, doc stackDocument) error { + var b strings.Builder + fmt.Fprintf(&b, "%s is up at %s\n", doc.Stack.Project, doc.Stack.URL) + var rows [][]string + for _, s := range doc.Services { + route := "" + if s.Route != "" { + route = "route " + s.Route + } + rows = append(rows, []string{s.Name, s.URL, route}) + } + b.WriteString(table(" ", rows)) + b.WriteString("Wait for the verdict with: siding status --wait\n") + _, err := fmt.Fprint(cmd.OutOrStdout(), b.String()) + return err +} diff --git a/internal/cli/up_test.go b/internal/cli/up_test.go new file mode 100644 index 0000000..abe7133 --- /dev/null +++ b/internal/cli/up_test.go @@ -0,0 +1,163 @@ +package cli + +import ( + "os" + "os/exec" + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/testutil" +) + +func TestUp(t *testing.T) { + t.Parallel() + + t.Run("starts the stack", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + before, _ := os.ReadFile(filepath.Join(e.repo, "compose.yaml")) + code, stdout, stderr := e.run("up", "--json", "--", "--build", "web") + if code != exitOK || stderr != "" { + t.Fatalf("exit %d, stderr %q", code, stderr) + } + testutil.Golden(t, filepath.Join("testdata", "json", "stack.json"), []byte(stdout)) + + runs := e.compose.recorded() + if len(runs) != 1 { + t.Fatalf("compose runs %+v", runs) + } + r := runs[0] + wantFiles := []string{filepath.Join(e.repo, "compose.yaml"), filepath.Join(e.repo, ".siding", "override.yml")} + if r.inv.Project != "shop-main" || !slices.Equal(r.inv.Files, wantFiles) || + !slices.Equal(r.args, []string{"up", "-d", "--remove-orphans", "--build", "web"}) { + t.Errorf("compose ran %+v", r) + } + override, err := os.ReadFile(wantFiles[1]) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(override), "ports: !reset []") || !strings.Contains(string(override), "Host(`shop.main.localhost`) && PathPrefix(`/api`)") { + t.Errorf("override:\n%s", override) + } + after, _ := os.ReadFile(filepath.Join(e.repo, "compose.yaml")) + if string(after) != string(before) { + t.Error("the compose file changed") + } + if out, err := exec.Command("git", "-C", e.repo, "status", "--porcelain").Output(); err != nil || len(out) != 0 { + t.Errorf("the worktree is dirty after up: %q, %v", out, err) + } + }) + + t.Run("text", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(8080) + code, stdout, _ := e.run("up") + want := `shop-main is up at http://shop.main.localhost:8080/ + api http://api.shop.main.localhost:8080/ route /api + mail http://mail.shop.main.localhost:8080/ + web http://web.shop.main.localhost:8080/ route / +Wait for the verdict with: siding status --wait +` + if code != exitOK || stdout != want { + t.Errorf("exit %d, stdout:\n%s\nwant:\n%s", code, stdout, want) + } + }) + + t.Run("agent", func(t *testing.T) { + t.Parallel() + for _, tt := range []struct { + name, env, flag, want string + }{ + {name: "from SIDING_AGENT", env: "claude", want: `"dev.siding.agent": "claude"`}, + {name: "the flag wins", env: "claude", flag: "codex", want: `"dev.siding.agent": "codex"`}, + {name: "none", want: ""}, + } { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + e.envVars["SIDING_AGENT"] = tt.env + args := []string{"up"} + if tt.flag != "" { + args = append(args, "--agent", tt.flag) + } + if code, _, stderr := e.run(args...); code != exitOK { + t.Fatalf("exit %d: %s", code, stderr) + } + override, _ := os.ReadFile(filepath.Join(e.repo, ".siding", "override.yml")) + got := strings.Contains(string(override), "dev.siding.agent") + if (tt.want == "") == got || (tt.want != "" && !strings.Contains(string(override), tt.want)) { + t.Errorf("override has agent %v, want %q", got, tt.want) + } + }) + } + }) + + t.Run("repo override", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + if code, stdout, _ := e.run("up", "--repo", "store"); code != exitOK || !strings.HasPrefix(stdout, "store-main is up at http://store.main.localhost/") { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) + + failures := []struct { + name string + setup func(e *env) + code string + }{ + {name: "proxy not running", setup: func(*env) {}, code: fail.ProxyNotRunning}, + {name: "outside a repository", setup: func(e *env) { e.proxyUp(80); e.opts.Dir = t.TempDir() }, code: fail.NotAGitRepository}, + {name: "detached HEAD", setup: func(e *env) { e.proxyUp(80); e.git("checkout", "-q", "--detach") }, code: fail.NoBranch}, + {name: "no compose file", setup: func(e *env) { + e.proxyUp(80) + _ = os.Remove(filepath.Join(e.repo, "compose.yaml")) + }, code: fail.ComposeFileNotFound}, + {name: "slug collision", setup: func(e *env) { + e.proxyUp(80) + other := e.id() + other.Branch, other.Worktree = "main", "/src/elsewhere" + e.docker.add(container(other, "web", "running", "", 0)) + }, code: fail.SlugCollision}, + {name: "compose fails", setup: func(e *env) { + e.proxyUp(80) + e.compose.runErr = fail.New(fail.ComposeFailed, "docker compose up -d failed: pull access denied for nope") + }, code: fail.ComposeFailed}, + {name: "two services on one path", setup: func(e *env) { + e.proxyUp(80) + e.compose.config = `{"services": {"a": {"expose": ["80"], "labels": {"dev.siding.route": "/"}}, "b": {"expose": ["80"], "labels": {"dev.siding.route": "/"}}}}` + }, code: fail.InvalidRoute}, + } + for _, tt := range failures { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + e := newEnv(t) + tt.setup(e) + code, stdout, _ := e.run("up", "--json") + if code != exitFailed || !strings.Contains(stdout, `"code": "`+tt.code+`"`) { + t.Errorf("exit %d, stdout %s; want code %s", code, stdout, tt.code) + } + if tt.code != fail.ComposeFailed && len(e.compose.recorded()) != 0 { + t.Errorf("compose ran although up failed: %+v", e.compose.recorded()) + } + }) + } + + t.Run("a plain Compose project of the same name is adopted", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + plain := container(e.id(), "web", "running", "", 0) + plain.Labels = map[string]string{"com.docker.compose.project": "shop-main", "com.docker.compose.service": "web"} + e.docker.add(plain) + if code, _, stderr := e.run("up"); code != exitOK { + t.Errorf("exit %d: %s", code, stderr) + } + }) +} diff --git a/internal/compose/compose.go b/internal/compose/compose.go new file mode 100644 index 0000000..043092c --- /dev/null +++ b/internal/compose/compose.go @@ -0,0 +1,165 @@ +package compose + +import ( + "bytes" + "context" + "errors" + "fmt" + "io" + "os/exec" + "slices" + "strings" + "sync" + + "github.com/tweinmann/siding/internal/fail" +) + +// MinVersion is the oldest Compose siding works with: `ports: !reset []` needs 2.24. +const MinVersion = "2.24" + +// Invocation names a Compose project and the files it is made of. +type Invocation struct { + // Dir is the working directory; Compose's project directory is the first file's. + Dir string + Project string + // Files are passed as -f, in order. Without files, Compose acts on the project by name. + Files []string +} + +func (inv Invocation) args(rest ...string) []string { + args := []string{"compose", "-p", inv.Project} + for _, f := range inv.Files { + args = append(args, "-f", f) + } + return append(args, rest...) +} + +// CLI runs docker compose. Its zero value uses "docker" from PATH. +type CLI struct { + // Binary is the docker executable; empty means "docker". + Binary string + + once sync.Once + versionErr error +} + +func (c *CLI) bin() string { + if c.Binary == "" { + return "docker" + } + return c.Binary +} + +// checkVersion fails with ComposeFailed unless Compose is installed and at least MinVersion. +// It runs once per CLI. +func (c *CLI) checkVersion(ctx context.Context) error { + c.once.Do(func() { + out, err := exec.CommandContext(ctx, c.bin(), "compose", "version", "--short").Output() + if err != nil { + c.versionErr = fail.New(fail.ComposeFailed, "docker compose is not available: %v", err). + WithHint("install the Docker Compose plugin %s or later", MinVersion) + return + } + v := strings.TrimPrefix(strings.TrimSpace(string(out)), "v") + if !AtLeast(v, MinVersion) { + c.versionErr = fail.New(fail.ComposeFailed, "docker compose %s is too old", v). + WithHint("siding needs Docker Compose %s or later", MinVersion) + } + }) + return c.versionErr +} + +// AtLeast compares dotted version numbers: AtLeast("2.24.1", "2.24") is true. +func AtLeast(have, want string) bool { + h, w := strings.Split(have, "."), strings.Split(want, ".") + for i := range w { + var hn, wn int + if i < len(h) { + _, _ = fmt.Sscanf(h[i], "%d", &hn) + } + _, _ = fmt.Sscanf(w[i], "%d", &wn) + if hn != wn { + return hn > wn + } + } + return true +} + +// Config returns the output of docker compose config --format json. +func (c *CLI) Config(ctx context.Context, inv Invocation) ([]byte, error) { + if err := c.checkVersion(ctx); err != nil { + return nil, err + } + var out, errOut bytes.Buffer + if err := c.exec(ctx, inv, inv.args("config", "--format", "json"), &out, &errOut); err != nil { + return nil, composeError("read the compose configuration", errOut.String(), err) + } + return out.Bytes(), nil +} + +// Run runs docker compose with args, streaming its output. On failure the error carries the +// last lines Compose printed on stderr. +func (c *CLI) Run(ctx context.Context, inv Invocation, args []string, stdout, stderr io.Writer) error { + if err := c.checkVersion(ctx); err != nil { + return err + } + tail := &tailBuffer{max: 20} + if err := c.exec(ctx, inv, inv.args(args...), stdout, io.MultiWriter(stderr, tail)); err != nil { + if ctx.Err() != nil { + return ctx.Err() // interrupted, e.g. Ctrl-C during logs --follow + } + return composeError("docker compose "+strings.Join(args, " "), tail.String(), err) + } + return nil +} + +func (c *CLI) exec(ctx context.Context, inv Invocation, args []string, stdout, stderr io.Writer) error { + cmd := exec.CommandContext(ctx, c.bin(), args...) + cmd.Dir = inv.Dir + // No stdin: a question from Compose fails instead of waiting for an answer nobody gives. + cmd.Stdin = nil + cmd.Stdout = stdout + cmd.Stderr = stderr + return cmd.Run() +} + +func composeError(what, output string, err error) error { + var exit *exec.ExitError + if !errors.As(err, &exit) { + return fail.New(fail.ComposeFailed, "%s: %v", what, err) + } + output = strings.TrimSpace(output) + if output == "" { + return fail.New(fail.ComposeFailed, "%s failed with exit code %d", what, exit.ExitCode()) + } + return fail.New(fail.ComposeFailed, "%s failed: %s", what, output) +} + +// tailBuffer keeps the last max lines written to it. +type tailBuffer struct { + max int + lines []string + partial string +} + +func (t *tailBuffer) Write(p []byte) (int, error) { + s := t.partial + string(p) + parts := strings.Split(s, "\n") + t.partial = parts[len(parts)-1] + t.lines = append(t.lines, parts[:len(parts)-1]...) + if len(t.lines) > t.max { + t.lines = t.lines[len(t.lines)-t.max:] + } + return len(p), nil +} + +func (t *tailBuffer) String() string { + lines := slices.Clone(t.lines) + if t.partial != "" { + lines = append(lines, t.partial) + } + if len(lines) > t.max { + lines = lines[len(lines)-t.max:] + } + return strings.Join(lines, "\n") +} diff --git a/internal/compose/config.go b/internal/compose/config.go new file mode 100644 index 0000000..753f6b5 --- /dev/null +++ b/internal/compose/config.go @@ -0,0 +1,188 @@ +// Package compose runs docker compose and reads the normalised configuration it prints. siding +// never parses a compose file itself: interpolation, profiles, include and extends are +// Compose's business. +package compose + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "slices" + "sort" + "strconv" + "strings" + + "github.com/tweinmann/siding/internal/fail" +) + +// FileNames are the compose files Compose itself looks for, in its order. +var FileNames = []string{"compose.yaml", "compose.yml", "docker-compose.yaml", "docker-compose.yml"} + +// FindFile returns the compose file of a worktree: file when given (relative to dir), else the +// first of FileNames in top. +func FindFile(dir, top, file string) (string, error) { + if file != "" { + if !filepath.IsAbs(file) { + file = filepath.Join(dir, file) + } + if _, err := os.Stat(file); err != nil { + return "", fail.New(fail.ComposeFileNotFound, "compose file %s not found", file) + } + return file, nil + } + for _, name := range FileNames { + path := filepath.Join(top, name) + if _, err := os.Stat(path); err == nil { + return path, nil + } + } + return "", fail.New(fail.ComposeFileNotFound, "no compose file in %s", top). + WithHint("siding looks for %s; name another one with --file", strings.Join(FileNames, ", ")) +} + +// Project is the part of a normalised compose configuration siding needs. +type Project struct { + Name string + Services map[string]Service + Networks map[string]Resource + Volumes map[string]Resource +} + +// ServiceNames returns the service names in the order Compose prints them (sorted). +func (p *Project) ServiceNames() []string { + names := make([]string, 0, len(p.Services)) + for n := range p.Services { + names = append(names, n) + } + sort.Strings(names) + return names +} + +// Service is one service of a Project. +type Service struct { + // Ports are the distinct container ports from ports (targets) and expose, ascending. + Ports []int + Labels map[string]string + Networks []string // keys of the service's networks, sorted + NetworkMode string + Environment map[string]string +} + +// Resource is a top-level network or volume. +type Resource struct { + Name string + External bool +} + +type rawProject struct { + Name string `json:"name"` + Services map[string]rawService `json:"services"` + Networks map[string]rawResource `json:"networks"` + Volumes map[string]rawResource `json:"volumes"` +} + +type rawService struct { + Ports []struct { + Target int `json:"target"` + Protocol string `json:"protocol"` + } `json:"ports"` + Expose []json.RawMessage `json:"expose"` + Labels map[string]string `json:"labels"` + Networks map[string]json.RawMessage `json:"networks"` + NetworkMode string `json:"network_mode"` + Environment map[string]*json.RawMessage `json:"environment"` +} + +type rawResource struct { + Name string `json:"name"` + External bool `json:"external"` +} + +// ParseConfig reads the output of docker compose config --format json. +func ParseConfig(data []byte) (*Project, error) { + var raw rawProject + if err := json.Unmarshal(data, &raw); err != nil { + return nil, fmt.Errorf("read compose config: %w", err) + } + p := &Project{Name: raw.Name, Services: map[string]Service{}, Networks: map[string]Resource{}, Volumes: map[string]Resource{}} + for name, rs := range raw.Services { + s := Service{Labels: rs.Labels, NetworkMode: rs.NetworkMode, Environment: map[string]string{}} + if s.Labels == nil { + s.Labels = map[string]string{} + } + ports := map[int]bool{} + for _, port := range rs.Ports { + if port.Protocol == "" || port.Protocol == "tcp" { + ports[port.Target] = true + } + } + for _, e := range rs.Expose { + expanded, err := parseExpose(e) + if err != nil { + return nil, fmt.Errorf("service %s: %w", name, err) + } + for _, n := range expanded { + ports[n] = true + } + } + for n := range ports { + s.Ports = append(s.Ports, n) + } + slices.Sort(s.Ports) + for n := range rs.Networks { + s.Networks = append(s.Networks, n) + } + slices.Sort(s.Networks) + for k, v := range rs.Environment { + if v == nil { + continue + } + var str string + if err := json.Unmarshal(*v, &str); err == nil { + s.Environment[k] = str + } + } + p.Services[name] = s + } + for k, r := range raw.Networks { + p.Networks[k] = Resource(r) + } + for k, r := range raw.Volumes { + p.Volumes[k] = Resource(r) + } + return p, nil +} + +// parseExpose expands one expose entry ("3000", "3000/tcp", "9000-9001", or a number) into +// TCP ports. +func parseExpose(raw json.RawMessage) ([]int, error) { + var s string + if err := json.Unmarshal(raw, &s); err != nil { + var n int + if err := json.Unmarshal(raw, &n); err != nil { + return nil, fmt.Errorf("expose entry %s", raw) + } + return []int{n}, nil + } + spec, proto, _ := strings.Cut(s, "/") + if proto != "" && proto != "tcp" { + return nil, nil + } + lo, hi, isRange := strings.Cut(spec, "-") + first, err := strconv.Atoi(lo) + if err != nil { + return nil, fmt.Errorf("expose entry %q", s) + } + last := first + if isRange { + if last, err = strconv.Atoi(hi); err != nil || last < first { + return nil, fmt.Errorf("expose entry %q", s) + } + } + // Only the number of ports matters (one makes an HTTP service), so a range is its ends. + if last > first { + return []int{first, last}, nil + } + return []int{first}, nil +} diff --git a/internal/compose/config_test.go b/internal/compose/config_test.go new file mode 100644 index 0000000..68152f1 --- /dev/null +++ b/internal/compose/config_test.go @@ -0,0 +1,126 @@ +package compose + +import ( + "errors" + "os" + "path/filepath" + "reflect" + "testing" + + "github.com/tweinmann/siding/internal/fail" +) + +// testdata/routing/config.json is what `docker compose -p shop-main config --format json` +// prints for testdata/routing/compose.yaml with the Compose pinned in the devcontainer. After a +// Compose upgrade, capture it again in that directory. +func TestParseConfig(t *testing.T) { + t.Parallel() + data, err := os.ReadFile(filepath.Join("testdata", "routing", "config.json")) + if err != nil { + t.Fatal(err) + } + p, err := ParseConfig(data) + if err != nil { + t.Fatal(err) + } + if p.Name != "shop-main" { + t.Errorf("name %q", p.Name) + } + if got, want := p.ServiceNames(), []string{"admin", "api", "db", "mail", "tools", "web", "worker"}; !reflect.DeepEqual(got, want) { + t.Errorf("services %v, want %v", got, want) + } + services := []struct { + name string + ports []int + networks []string + networkMode string + route string + }{ + {name: "web", ports: []int{3000}, networks: []string{"default"}, route: "/"}, + {name: "api", ports: []int{8080, 9229}, networks: []string{"backend", "default"}, route: "/api"}, + {name: "mail", ports: []int{8025}, networks: []string{"default"}}, + {name: "admin", ports: []int{80}, networks: []string{"default"}, route: "false"}, + {name: "db", ports: []int{5432}, networks: []string{"backend"}}, + {name: "worker", networks: []string{"default"}}, + {name: "tools", ports: []int{7000, 7001}, networkMode: "host"}, + } + for _, want := range services { + s := p.Services[want.name] + if !reflect.DeepEqual(s.Ports, want.ports) || !reflect.DeepEqual(s.Networks, want.networks) || + s.NetworkMode != want.networkMode || s.Labels["dev.siding.route"] != want.route { + t.Errorf("%s: ports %v, networks %v, network_mode %q, route %q; want %v, %v, %q, %q", want.name, + s.Ports, s.Networks, s.NetworkMode, s.Labels["dev.siding.route"], want.ports, want.networks, want.networkMode, want.route) + } + } + if got := p.Services["api"].Environment["API_TOKEN"]; got != "not-a-real-api-token" { + t.Errorf("env_file value %q, want it inlined into environment", got) + } + if got := p.Services["web"].Environment["DB_URL"]; got != "postgres://app:not-a-real-token@db/app" { + t.Errorf("interpolated value %q", got) + } + if got := p.Volumes["shared"]; !got.External || got.Name != "shared-cache" { + t.Errorf("external volume %+v", got) + } + if got := p.Volumes["data"]; got.External || got.Name != "shop-main_data" { + t.Errorf("volume %+v", got) + } + if got := p.Networks["backend"]; got.External || got.Name != "shop-main_backend" { + t.Errorf("network %+v", got) + } +} + +func TestFindFile(t *testing.T) { + t.Parallel() + top := t.TempDir() + if _, err := FindFile(top, top, ""); codeOf(err) != fail.ComposeFileNotFound { + t.Fatalf("empty worktree: %v", err) + } + for _, name := range []string{"docker-compose.yml", "compose.yaml"} { + if err := os.WriteFile(filepath.Join(top, name), nil, 0o644); err != nil { + t.Fatal(err) + } + } + if got, err := FindFile(top, top, ""); err != nil || got != filepath.Join(top, "compose.yaml") { + t.Errorf("got %q, %v; want compose.yaml first", got, err) + } + sub := filepath.Join(top, "sub") + if err := os.MkdirAll(sub, 0o755); err != nil { + t.Fatal(err) + } + if got, err := FindFile(sub, top, "../docker-compose.yml"); err != nil || got != filepath.Join(top, "docker-compose.yml") { + t.Errorf("--file relative to the current directory: %q, %v", got, err) + } + if _, err := FindFile(top, top, "nope.yml"); codeOf(err) != fail.ComposeFileNotFound { + t.Errorf("missing --file: %v", err) + } +} + +func TestAtLeast(t *testing.T) { + t.Parallel() + for _, tt := range []struct { + have string + want bool + }{{"2.24.0", true}, {"2.24", true}, {"2.23.9", false}, {"5.6.1", true}, {"2.3", false}, {"2.100", true}} { + if got := AtLeast(tt.have, MinVersion); got != tt.want { + t.Errorf("AtLeast(%q, %q) = %v, want %v", tt.have, MinVersion, got, tt.want) + } + } +} + +func TestTailBuffer(t *testing.T) { + t.Parallel() + b := &tailBuffer{max: 2} + _, _ = b.Write([]byte("one\ntwo\nthr")) + _, _ = b.Write([]byte("ee\nfour")) + if got := b.String(); got != "three\nfour" { + t.Errorf("got %q", got) + } +} + +func codeOf(err error) string { + var e *fail.Error + if errors.As(err, &e) { + return e.Code + } + return "" +} diff --git a/internal/compose/testdata/routing/.env b/internal/compose/testdata/routing/.env new file mode 100644 index 0000000..c9d92b7 --- /dev/null +++ b/internal/compose/testdata/routing/.env @@ -0,0 +1 @@ +DB_PASSWORD=not-a-real-token diff --git a/internal/compose/testdata/routing/api.env b/internal/compose/testdata/routing/api.env new file mode 100644 index 0000000..a17f2ca --- /dev/null +++ b/internal/compose/testdata/routing/api.env @@ -0,0 +1 @@ +API_TOKEN=not-a-real-api-token diff --git a/internal/compose/testdata/routing/compose.yaml b/internal/compose/testdata/routing/compose.yaml new file mode 100644 index 0000000..466b118 --- /dev/null +++ b/internal/compose/testdata/routing/compose.yaml @@ -0,0 +1,40 @@ +# Every case the parser and the routing table must handle. +services: + web: + image: busybox:1.37.0 + ports: ["3000:3000"] + labels: [dev.siding.route=/] + healthcheck: {test: ["CMD", "true"]} + environment: + NODE_ENV: development + DB_URL: postgres://app:${DB_PASSWORD}@db/app + api: + image: busybox:1.37.0 + expose: ["8080", "9229"] + env_file: api.env + networks: [backend, default] + labels: {dev.siding.route: /api, dev.siding.port: "8080"} + mail: + image: busybox:1.37.0 + expose: ["8025/tcp", "1025/udp"] + admin: + image: busybox:1.37.0 + ports: ["127.0.0.1:9000:80"] + labels: {dev.siding.route: "false"} + db: + image: busybox:1.37.0 + ports: ["5432:5432"] + volumes: [data:/var/lib/data, shared:/shared] + networks: + backend: {aliases: [database]} + worker: + image: busybox:1.37.0 + tools: + image: busybox:1.37.0 + network_mode: host + expose: ["7000-7001"] +networks: + backend: {} +volumes: + data: {} + shared: {external: true, name: shared-cache} diff --git a/internal/compose/testdata/routing/config.json b/internal/compose/testdata/routing/config.json new file mode 100644 index 0000000..bcff27a --- /dev/null +++ b/internal/compose/testdata/routing/config.json @@ -0,0 +1,156 @@ +{ + "name": "shop-main", + "networks": { + "backend": { + "name": "shop-main_backend", + "ipam": {} + }, + "default": { + "name": "shop-main_default", + "ipam": {} + } + }, + "services": { + "admin": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "labels": { + "dev.siding.route": "false" + }, + "networks": { + "default": null + }, + "ports": [ + { + "mode": "ingress", + "host_ip": "127.0.0.1", + "target": 80, + "published": "9000", + "protocol": "tcp" + } + ] + }, + "api": { + "command": null, + "entrypoint": null, + "environment": { + "API_TOKEN": "not-a-real-api-token" + }, + "image": "busybox:1.37.0", + "labels": { + "dev.siding.port": "8080", + "dev.siding.route": "/api" + }, + "networks": { + "backend": null, + "default": null + }, + "expose": [ + "8080", + "9229" + ] + }, + "db": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "networks": { + "backend": { + "aliases": [ + "database" + ] + } + }, + "volumes": [ + { + "type": "volume", + "source": "data", + "target": "/var/lib/data", + "volume": {} + }, + { + "type": "volume", + "source": "shared", + "target": "/shared", + "volume": {} + } + ], + "ports": [ + { + "mode": "ingress", + "target": 5432, + "published": "5432", + "protocol": "tcp" + } + ] + }, + "mail": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "networks": { + "default": null + }, + "expose": [ + "8025/tcp", + "1025/udp" + ] + }, + "tools": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "network_mode": "host", + "expose": [ + "7000-7001" + ] + }, + "web": { + "command": null, + "entrypoint": null, + "environment": { + "DB_URL": "postgres://app:not-a-real-token@db/app", + "NODE_ENV": "development" + }, + "image": "busybox:1.37.0", + "labels": { + "dev.siding.route": "/" + }, + "networks": { + "default": null + }, + "healthcheck": { + "test": [ + "CMD", + "true" + ] + }, + "ports": [ + { + "mode": "ingress", + "target": 3000, + "published": "3000", + "protocol": "tcp" + } + ] + }, + "worker": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "networks": { + "default": null + } + } + }, + "volumes": { + "data": { + "name": "shop-main_data" + }, + "shared": { + "name": "shared-cache", + "external": true + } + } +} \ No newline at end of file diff --git a/internal/docker/docker.go b/internal/docker/docker.go new file mode 100644 index 0000000..26b192d --- /dev/null +++ b/internal/docker/docker.go @@ -0,0 +1,324 @@ +// Package docker reads the state of stacks from the Docker Engine API. Docker is siding's +// database: containers, networks and volumes with their labels are all it knows. +package docker + +import ( + "bufio" + "bytes" + "context" + "encoding/json" + "fmt" + "os" + "os/exec" + "strconv" + "strings" + "sync" + "time" + + cerrdefs "github.com/containerd/errdefs" + "github.com/moby/moby/api/pkg/stdcopy" + "github.com/moby/moby/api/types/container" + "github.com/moby/moby/client" + + "github.com/tweinmann/siding/internal/fail" +) + +// Container is what siding knows about one container. +type Container struct { + ID string + Name string + Image string + Labels map[string]string + // State is one of created, running, paused, restarting, removing, exited or dead. + State string + // Health is "", starting, healthy or unhealthy. + Health string + // HealthOutput is the output of the last healthcheck. + HealthOutput string + ExitCode int + RestartCount int + OOMKilled bool + Ports []Port +} + +// Port is a published port. +type Port struct { + HostIP string + HostPort int + ContainerPort int +} + +// Network is a Docker network. +type Network struct { + Name string + Labels map[string]string +} + +// Volume is a Docker volume. +type Volume struct { + Name string + Labels map[string]string +} + +// Client talks to the Docker daemon that docker and docker compose use. +type Client struct { + once sync.Once + api *client.Client + err error +} + +// connect resolves the daemon the docker CLI uses: DOCKER_HOST, else the current Docker +// context's endpoint, so siding and docker compose always see the same daemon. +func (c *Client) connect() (*client.Client, error) { + c.once.Do(func() { + opts := []client.Opt{client.FromEnv} + if os.Getenv("DOCKER_HOST") == "" { + if out, err := exec.Command("docker", "context", "inspect", "--format", "{{.Endpoints.docker.Host}}").Output(); err == nil { + if host := strings.TrimSpace(string(out)); host != "" { + opts = append(opts, client.WithHost(host)) + } + } + } + c.api, c.err = client.New(opts...) + if c.err != nil { + c.err = unavailable(c.err) + } + }) + return c.api, c.err +} + +func unavailable(err error) error { + return fail.New(fail.DockerUnavailable, "cannot reach the Docker daemon: %v", err). + WithHint("start Docker, or check DOCKER_HOST and docker context") +} + +// Ping fails with DockerUnavailable when the daemon cannot be reached. +func (c *Client) Ping(ctx context.Context) error { + api, err := c.connect() + if err != nil { + return err + } + if _, err := api.Ping(ctx, client.PingOptions{}); err != nil { + return unavailable(err) + } + return nil +} + +func labelFilters(labels map[string]string) client.Filters { + f := client.Filters{} + for k, v := range labels { + if v == "" { + f.Add("label", k) + } else { + f.Add("label", k+"="+v) + } + } + return f +} + +// Containers returns every container, running or not, that carries all the given labels; an +// empty value matches any value. +func (c *Client) Containers(ctx context.Context, labels map[string]string) ([]Container, error) { + api, err := c.connect() + if err != nil { + return nil, err + } + list, err := api.ContainerList(ctx, client.ContainerListOptions{All: true, Filters: labelFilters(labels)}) + if err != nil { + return nil, unavailable(err) + } + out := make([]Container, 0, len(list.Items)) + for _, s := range list.Items { + ct, err := c.inspect(ctx, api, s.ID) + if err != nil { + if cerrdefs.IsNotFound(err) { + continue // removed meanwhile + } + return nil, err + } + out = append(out, ct) + } + return out, nil +} + +// Container returns the container with this name, or nil when there is none. +func (c *Client) Container(ctx context.Context, name string) (*Container, error) { + api, err := c.connect() + if err != nil { + return nil, err + } + ct, err := c.inspect(ctx, api, name) + if cerrdefs.IsNotFound(err) { + return nil, nil + } + if err != nil { + return nil, err + } + return &ct, nil +} + +func (c *Client) inspect(ctx context.Context, api *client.Client, id string) (Container, error) { + res, err := api.ContainerInspect(ctx, id, client.ContainerInspectOptions{}) + if err != nil { + return Container{}, err + } + return fromInspect(res.Container), nil +} + +func fromInspect(in container.InspectResponse) Container { + ct := Container{ID: in.ID, Name: strings.TrimPrefix(in.Name, "/"), RestartCount: in.RestartCount} + if in.Config != nil { + ct.Image = in.Config.Image + ct.Labels = in.Config.Labels + } + if ct.Labels == nil { + ct.Labels = map[string]string{} + } + if s := in.State; s != nil { + ct.State = string(s.Status) + ct.ExitCode = s.ExitCode + ct.OOMKilled = s.OOMKilled + if h := s.Health; h != nil { + ct.Health = string(h.Status) + if n := len(h.Log); n > 0 && h.Log[n-1] != nil { + ct.HealthOutput = strings.TrimSpace(h.Log[n-1].Output) + } + } + } + if in.NetworkSettings != nil { + for port, bindings := range in.NetworkSettings.Ports { + for _, b := range bindings { + hostPort, err := strconv.Atoi(b.HostPort) + if err != nil { + continue + } + ip := "" + if b.HostIP.IsValid() { + ip = b.HostIP.String() + } + ct.Ports = append(ct.Ports, Port{HostIP: ip, HostPort: hostPort, ContainerPort: int(port.Num())}) + } + } + } + return ct +} + +// Networks returns the networks that carry all the given labels. +func (c *Client) Networks(ctx context.Context, labels map[string]string) ([]Network, error) { + api, err := c.connect() + if err != nil { + return nil, err + } + list, err := api.NetworkList(ctx, client.NetworkListOptions{Filters: labelFilters(labels)}) + if err != nil { + return nil, unavailable(err) + } + out := make([]Network, 0, len(list.Items)) + for _, n := range list.Items { + out = append(out, Network{Name: n.Name, Labels: n.Labels}) + } + return out, nil +} + +// EnsureNetwork creates the network unless it exists and reports whether it created it. +func (c *Client) EnsureNetwork(ctx context.Context, name string, labels map[string]string) (bool, error) { + api, err := c.connect() + if err != nil { + return false, err + } + if _, err := api.NetworkInspect(ctx, name, client.NetworkInspectOptions{}); err == nil { + return false, nil + } else if !cerrdefs.IsNotFound(err) { + return false, unavailable(err) + } + if _, err := api.NetworkCreate(ctx, name, client.NetworkCreateOptions{Driver: "bridge", Labels: labels}); err != nil { + return false, fmt.Errorf("create network %s: %w", name, err) + } + return true, nil +} + +// Volumes returns the volumes that carry all the given labels. +func (c *Client) Volumes(ctx context.Context, labels map[string]string) ([]Volume, error) { + api, err := c.connect() + if err != nil { + return nil, err + } + list, err := api.VolumeList(ctx, client.VolumeListOptions{Filters: labelFilters(labels)}) + if err != nil { + return nil, unavailable(err) + } + out := make([]Volume, 0, len(list.Items)) + for _, v := range list.Items { + out = append(out, Volume{Name: v.Name, Labels: v.Labels}) + } + return out, nil +} + +// Logs returns the last tail lines a container wrote to stdout and stderr, in order. +func (c *Client) Logs(ctx context.Context, id string, tail int) ([]string, error) { + api, err := c.connect() + if err != nil { + return nil, err + } + rc, err := api.ContainerLogs(ctx, id, client.ContainerLogsOptions{ShowStdout: true, ShowStderr: true, Tail: strconv.Itoa(tail)}) + if err != nil { + return nil, fmt.Errorf("logs of %s: %w", id, err) + } + defer func() { _ = rc.Close() }() + // Both streams go into one buffer, so lines keep the order the daemon sent them in. A + // container with a TTY sends a plain stream, not a multiplexed one. + var buf bytes.Buffer + raw, err := readAll(rc) + if err != nil { + return nil, err + } + if _, err := stdcopy.StdCopy(&buf, &buf, bytes.NewReader(raw)); err != nil { + buf.Reset() + buf.Write(raw) + } + var lines []string + sc := bufio.NewScanner(&buf) + sc.Buffer(make([]byte, 64*1024), 1024*1024) + for sc.Scan() { + lines = append(lines, strings.TrimRight(sc.Text(), "\r")) + } + return lines, sc.Err() +} + +func readAll(r interface{ Read([]byte) (int, error) }) ([]byte, error) { + var b bytes.Buffer + _, err := b.ReadFrom(r) + return b.Bytes(), err +} + +// Memory returns the memory a running container uses now, counted as docker stats counts it +// (usage without the inactive page cache). +func (c *Client) Memory(ctx context.Context, id string) (uint64, error) { + api, err := c.connect() + if err != nil { + return 0, err + } + ctx, cancel := context.WithTimeout(ctx, 10*time.Second) + defer cancel() + res, err := api.ContainerStats(ctx, id, client.ContainerStatsOptions{Stream: false}) + if err != nil { + return 0, fmt.Errorf("stats of %s: %w", id, err) + } + defer func() { _ = res.Body.Close() }() + var stats container.StatsResponse + if err := json.NewDecoder(res.Body).Decode(&stats); err != nil { + return 0, fmt.Errorf("stats of %s: %w", id, err) + } + return MemoryUsage(stats.MemoryStats.Usage, stats.MemoryStats.Stats), nil +} + +// MemoryUsage subtracts the inactive page cache from usage: inactive_file on cgroup v2, +// total_inactive_file on cgroup v1. +func MemoryUsage(usage uint64, stats map[string]uint64) uint64 { + for _, key := range []string{"inactive_file", "total_inactive_file"} { + if v, ok := stats[key]; ok && v < usage { + return usage - v + } + } + return usage +} diff --git a/internal/docker/docker_test.go b/internal/docker/docker_test.go new file mode 100644 index 0000000..e9a35f2 --- /dev/null +++ b/internal/docker/docker_test.go @@ -0,0 +1,64 @@ +package docker + +import ( + "net/netip" + "reflect" + "testing" + + "github.com/moby/moby/api/types/container" + "github.com/moby/moby/api/types/network" +) + +func TestFromInspect(t *testing.T) { + t.Parallel() + port, err := network.ParsePort("80/tcp") + if err != nil { + t.Fatal(err) + } + in := container.InspectResponse{ + ID: "abc", + Name: "/siding-proxy", + RestartCount: 2, + Config: &container.Config{Image: "traefik:v3.7.16", Labels: map[string]string{"dev.siding.machine": "m"}}, + State: &container.State{ + Status: container.StateRunning, + ExitCode: 0, + Health: &container.Health{Status: container.Unhealthy, Log: []*container.HealthcheckResult{ + {Output: "old"}, {Output: "connection refused\n"}, + }}, + }, + NetworkSettings: &container.NetworkSettings{Ports: network.PortMap{ + port: {{HostIP: netip.MustParseAddr("127.0.0.1"), HostPort: "8080"}}, + }}, + } + want := Container{ + ID: "abc", Name: "siding-proxy", Image: "traefik:v3.7.16", Labels: map[string]string{"dev.siding.machine": "m"}, + State: "running", Health: "unhealthy", HealthOutput: "connection refused", RestartCount: 2, + Ports: []Port{{HostIP: "127.0.0.1", HostPort: 8080, ContainerPort: 80}}, + } + if got := fromInspect(in); !reflect.DeepEqual(got, want) { + t.Errorf("got %+v\nwant %+v", got, want) + } + if got := fromInspect(container.InspectResponse{}); got.Labels == nil { + t.Error("labels must never be nil") + } +} + +func TestMemoryUsage(t *testing.T) { + t.Parallel() + for _, tt := range []struct { + name string + usage uint64 + stats map[string]uint64 + want uint64 + }{ + {name: "cgroup v2", usage: 1000, stats: map[string]uint64{"inactive_file": 300}, want: 700}, + {name: "cgroup v1", usage: 1000, stats: map[string]uint64{"total_inactive_file": 100}, want: 900}, + {name: "no stats", usage: 1000, want: 1000}, + {name: "cache larger than usage", usage: 10, stats: map[string]uint64{"inactive_file": 20}, want: 10}, + } { + if got := MemoryUsage(tt.usage, tt.stats); got != tt.want { + t.Errorf("%s: %d, want %d", tt.name, got, tt.want) + } + } +} diff --git a/internal/e2e/e2e_test.go b/internal/e2e/e2e_test.go new file mode 100644 index 0000000..3a28e4c --- /dev/null +++ b/internal/e2e/e2e_test.go @@ -0,0 +1,309 @@ +//go:build docker + +// Package e2e runs siding's commands end to end against a real Docker daemon: the +// devcontainer's inner one, through just smoke-stacks. Everything it creates is named +// siding-test-* and removed by exact name; a developer's siding-proxy and network siding are +// never touched. +package e2e + +import ( + "bytes" + "context" + "encoding/json" + "io" + "net/http" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/tweinmann/siding/internal/cli" + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/probe" + "github.com/tweinmann/siding/internal/proxy" + "github.com/tweinmann/siding/internal/stack" +) + +const ( + proxyPort = 18080 + addr = "127.0.0.1:18080" +) + +var testProxy = proxy.Config{ + Project: "siding-test-e2e-proxy", + Container: "siding-test-e2e-proxy", + Network: "siding-test-e2e", + Image: proxy.Image, + DefaultPort: proxyPort, +} + +// A busybox httpd per service answers with the service's name at / and at /api. +const shopCompose = `services: + web: + image: busybox:1.37.0 + command: ["sh", "-c", "mkdir -p /www && echo web > /www/index.html && exec httpd -f -p 80 -h /www"] + ports: ["18081:80"] + labels: {dev.siding.route: "/"} + volumes: [data:/data] + healthcheck: {test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1/"], interval: 1s, retries: 30} + api: + image: busybox:1.37.0 + command: ["sh", "-c", "mkdir -p /www && echo api > /www/api && exec httpd -f -p 80 -h /www"] + expose: ["80"] + labels: {dev.siding.route: "/api"} +volumes: + data: {} +` + +// api crashes after logging a secret from .env; dead is routed but nothing listens on its port. +const brokenCompose = `services: + api: + image: busybox:1.37.0 + command: ["sh", "-c", "echo using token $$API_TOKEN; echo cannot connect to db; exit 1"] + environment: {API_TOKEN: "${API_TOKEN}"} + labels: {dev.siding.route: "/api"} + expose: ["80"] + dead: + image: busybox:1.37.0 + command: ["sleep", "600"] + expose: ["8081"] + labels: {dev.siding.route: "/"} +` + +type machine struct { + t *testing.T + home string +} + +func newMachine(t *testing.T) *machine { + t.Helper() + m := &machine{t: t, home: t.TempDir()} + t.Cleanup(func() { + // By exact name only. + run(t, "", "docker", "compose", "-p", testProxy.Project, "down", "-v", "--remove-orphans") + run(t, "", "docker", "network", "rm", testProxy.Network) + }) + return m +} + +// siding runs one command in dir and returns exit code and stdout; stderr goes to the log. +func (m *machine) siding(dir string, args ...string) (int, string) { + m.t.Helper() + opts := cli.Options{ + Dir: dir, Home: m.home, Getenv: os.Getenv, OSMachineID: stack.OSMachineID, + Git: git.Client{}, Compose: &compose.CLI{}, Docker: &docker.Client{}, + Probe: probe.HTTP{Header: stack.ServiceHeader}, Browser: noBrowser{}, Clock: cli.SystemClock{}, + Proxy: testProxy, + } + var out, errOut bytes.Buffer + code := cli.Execute(context.Background(), cli.New(opts), args, &out, &errOut) + m.t.Logf("siding %s: exit %d\n%s", strings.Join(args, " "), code, errOut.String()) + return code, out.String() +} + +type noBrowser struct{} + +func (noBrowser) Open(string) error { return nil } + +// repo creates a repository with a compose file on branch, one commit ahead of main, and +// removes its stack (project siding-test--) by exact name afterwards. +func repo(t *testing.T, name, composeFile, env, branch string) string { + t.Helper() + dir := filepath.Join(t.TempDir(), name) + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + gitIn(t, dir, "init", "-q", "-b", "main") + gitIn(t, dir, "commit", "-q", "--allow-empty", "-m", "first") + gitIn(t, dir, "switch", "-q", "-c", branch) + for file, content := range map[string]string{"compose.yaml": composeFile, ".env": env} { + if err := os.WriteFile(filepath.Join(dir, file), []byte(content), 0o644); err != nil { + t.Fatal(err) + } + } + gitIn(t, dir, "add", ".") + gitIn(t, dir, "commit", "-q", "-m", "app") + project := "siding-test-" + name + "-" + stack.Slug(branch) + t.Cleanup(func() { run(t, "", "docker", "compose", "-p", project, "down", "-v", "--remove-orphans") }) + return dir +} + +func gitIn(t *testing.T, dir string, args ...string) { + t.Helper() + cmd := exec.Command("git", append([]string{"-C", dir}, args...)...) + cmd.Env = append(os.Environ(), "GIT_CONFIG_GLOBAL=/dev/null", "GIT_CONFIG_NOSYSTEM=1", + "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@example.com", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@example.com") + if out, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("git %v: %v\n%s", args, err, out) + } +} + +// run runs a cleanup command and logs a failure instead of failing: the object may be gone. +func run(t *testing.T, dir string, name string, args ...string) string { + t.Helper() + cmd := exec.Command(name, args...) + cmd.Dir = dir + out, err := cmd.CombinedOutput() + if err != nil { + t.Logf("%s %v: %v\n%s", name, args, err, out) + } + return string(out) +} + +func get(t *testing.T, host, path string) (int, string, string) { + t.Helper() + req, _ := http.NewRequest(http.MethodGet, "http://"+addr+path, nil) + req.Host = host + resp, err := (&http.Client{Timeout: 5 * time.Second}).Do(req) + if err != nil { + t.Fatalf("GET %s%s: %v", host, path, err) + } + defer func() { _ = resp.Body.Close() }() + body, _ := io.ReadAll(resp.Body) + return resp.StatusCode, strings.TrimSpace(string(body)), resp.Header.Get(stack.ServiceHeader) +} + +type status struct { + Kind string `json:"kind"` + Verdict string `json:"verdict"` + Code string `json:"code"` + Failure struct { + Service, Reason, Hint string + Logs []string + } `json:"failure"` +} + +func decode(t *testing.T, stdout string) status { + t.Helper() + var s status + if err := json.Unmarshal([]byte(stdout), &s); err != nil { + t.Fatalf("not a document: %v\n%s", err, stdout) + } + return s +} + +func TestStacks(t *testing.T) { + m := newMachine(t) + + // setup, twice. + if code, out := m.siding(t.TempDir(), "setup", "--json"); code != 0 || !strings.Contains(out, `"state": "created"`) { + t.Fatalf("setup: exit %d\n%s", code, out) + } + if code, out := m.siding(t.TempDir(), "setup", "--json"); code != 0 || !strings.Contains(out, `"state": "unchanged"`) { + t.Fatalf("second setup: exit %d\n%s", code, out) + } + + shop := repo(t, "shop", shopCompose, "", "feature/x") + host := "siding-test-shop.feature-x.localhost" + + t.Run("up --wait is Ready", func(t *testing.T) { + code, out := m.siding(shop, "up", "--repo", "siding-test-shop", "--wait", "--timeout", "90s", "--json") + if s := decode(t, out); code != 0 || s.Verdict != "Ready" { + t.Fatalf("exit %d\n%s", code, out) + } + if got := run(t, shop, "git", "status", "--porcelain"); got != "" { + t.Errorf("the worktree is dirty after up: %q", got) + } + }) + + t.Run("routing by host and path", func(t *testing.T) { + for _, tt := range []struct { + host, path, body, service string + }{ + {host: host, path: "/", body: "web", service: "siding-test-shop-feature-x/web"}, + {host: host, path: "/api", body: "api", service: "siding-test-shop-feature-x/api"}, + {host: "web." + host, path: "/", body: "web", service: "siding-test-shop-feature-x/web"}, + } { + code, body, service := get(t, tt.host, tt.path) + if code != 200 || body != tt.body || service != tt.service { + t.Errorf("%s%s: %d %q, %s %q; want 200 %q from %s", tt.host, tt.path, code, body, stack.ServiceHeader, service, tt.body, tt.service) + } + } + // The app's own 404 carries the header; the proxy's 404 for an unknown host does not. + if code, _, service := get(t, host, "/api/nope"); code != 404 || service == "" { + t.Errorf("the app's 404: %d, header %q", code, service) + } + if code, _, service := get(t, "nope.localhost", "/"); code != 404 || service != "" { + t.Errorf("unknown host: %d, header %q", code, service) + } + }) + + t.Run("no published port", func(t *testing.T) { + out := run(t, "", "docker", "ps", "--filter", "label=com.docker.compose.project=siding-test-shop-feature-x", "--format", "{{.Ports}}") + if strings.Contains(out, "18081") || strings.Contains(out, "->") { + t.Errorf("published ports: %q", out) + } + }) + + t.Run("ps", func(t *testing.T) { + code, out := m.siding(t.TempDir(), "ps", "--json") + if code != 0 || !strings.Contains(out, `"project": "siding-test-shop-feature-x"`) || !strings.Contains(out, `"state": "running"`) { + t.Errorf("exit %d\n%s", code, out) + } + }) + + t.Run("a failure with its log tail", func(t *testing.T) { + broken := repo(t, "broken", brokenCompose, "API_TOKEN=not-a-real-token\n", "main-work") + code, out := m.siding(broken, "up", "--repo", "siding-test-broken", "--wait", "--timeout", "60s", "--json") + s := decode(t, out) + if code != 1 || s.Verdict != "Failed" || s.Failure.Service != "api" || s.Failure.Reason != "Exited" { + t.Fatalf("exit %d\n%s", code, out) + } + logs := strings.Join(s.Failure.Logs, "\n") + if !strings.Contains(logs, "cannot connect to db") || strings.Contains(out, "not-a-real-token") || !strings.Contains(logs, "using token ***") { + t.Errorf("log tail %q", logs) + } + // Nothing listens behind dead's route: the proxy answers 502 on its own. up ended at + // once with Failed, so give Traefik a moment to pick up dead's labels. + var code502 int + var service string + for i := 0; i < 20 && code502 != 502; i++ { + if i > 0 { + time.Sleep(500 * time.Millisecond) + } + code502, _, service = get(t, "siding-test-broken.main-work.localhost", "/") + } + resp := probe.Response{Status: code502, Service: service} + if code502 != 502 || resp.Answered("siding-test-broken-main-work/dead") { + t.Errorf("dead: status %d, header %q; want 502, not answered", code502, service) + } + t.Logf("%s on Traefik's own 502: %q", stack.ServiceHeader, service) + }) + + t.Run("down keeps the data", func(t *testing.T) { + if code, out := m.siding(shop, "down", "--json"); code != 0 || !strings.Contains(out, "siding-test-shop-feature-x-web-1") { + t.Fatalf("exit %d\n%s", code, out) + } + if out := run(t, "", "docker", "volume", "ls", "-q", "--filter", "name=siding-test-shop-feature-x_data"); !strings.Contains(out, "siding-test-shop-feature-x_data") { + t.Errorf("the volume is gone: %q", out) + } + if out := run(t, "", "docker", "ps", "-aq", "--filter", "label=com.docker.compose.project=siding-test-shop-feature-x"); strings.TrimSpace(out) != "" { + t.Errorf("containers left: %q", out) + } + }) + + t.Run("down --volumes keeps unmerged and dirty work", func(t *testing.T) { + if code, out := m.siding(shop, "up", "--repo", "siding-test-shop", "--wait", "--json"); code != 0 { + t.Fatalf("up again: exit %d\n%s", code, out) + } + if code, out := m.siding(shop, "down", "--volumes", "--json"); code != 1 || decode(t, out).Code != "UnmergedBranch" { + t.Errorf("unmerged: exit %d\n%s", code, out) + } + if err := os.WriteFile(filepath.Join(shop, "notes.txt"), []byte("unsaved"), 0o644); err != nil { + t.Fatal(err) + } + if code, out := m.siding(shop, "down", "--volumes", "--json"); code != 1 || decode(t, out).Code != "DirtyWorktree" { + t.Errorf("dirty: exit %d\n%s", code, out) + } + if code, out := m.siding(shop, "down", "--volumes", "--force", "--json"); code != 0 || !strings.Contains(out, "siding-test-shop-feature-x_data") { + t.Fatalf("forced: exit %d\n%s", code, out) + } + if out := run(t, "", "docker", "volume", "ls", "-q", "--filter", "name=siding-test-shop-feature-x_data"); strings.TrimSpace(out) != "" { + t.Errorf("the volume survived --force: %q", out) + } + }) +} diff --git a/internal/fail/fail.go b/internal/fail/fail.go new file mode 100644 index 0000000..f853cf3 --- /dev/null +++ b/internal/fail/fail.go @@ -0,0 +1,60 @@ +// Package fail holds the error type every siding command fails with: a stable code that agents +// may branch on, a message, and an optional hint on what to do next. Codes are part of the +// --json contract and are never renamed within an apiVersion. +package fail + +import "fmt" + +// Error is a failure with a stable code. +type Error struct { + Code string + Message string + Hint string +} + +func (e *Error) Error() string { return e.Message } + +// New returns an Error with a formatted message. +func New(code, format string, args ...any) *Error { + return &Error{Code: code, Message: fmt.Sprintf(format, args...)} +} + +// WithHint returns e with its hint set. +func (e *Error) WithHint(format string, args ...any) *Error { + e.Hint = fmt.Sprintf(format, args...) + return e +} + +// Error codes. The spec cli-contract lists them; add new ones there too. +const ( + // InvalidArguments: unknown command, flag or argument. + InvalidArguments = "InvalidArguments" + // NotAGitRepository: the current directory is not inside a git worktree. + NotAGitRepository = "NotAGitRepository" + // NoBranch: the worktree's HEAD is detached. + NoBranch = "NoBranch" + // ComposeFileNotFound: the worktree has no compose file. + ComposeFileNotFound = "ComposeFileNotFound" + // ComposeFailed: docker compose failed, or is missing or too old. + ComposeFailed = "ComposeFailed" + // DockerUnavailable: the Docker daemon cannot be reached. + DockerUnavailable = "DockerUnavailable" + // ProxyNotRunning: siding-proxy is not running. + ProxyNotRunning = "ProxyNotRunning" + // PortInUse: the proxy's port is held by another process. + PortInUse = "PortInUse" + // InvalidRoute: a dev.siding.route or dev.siding.port label cannot be used. + InvalidRoute = "InvalidRoute" + // AmbiguousPort: a routed service has several container ports and no dev.siding.port. + AmbiguousPort = "AmbiguousPort" + // SlugCollision: another branch or worktree already owns the Compose project. + SlugCollision = "SlugCollision" + // StackNotFound: the worktree has no stack. + StackNotFound = "StackNotFound" + // UnmergedBranch: removing data refused because the branch is not merged. + UnmergedBranch = "UnmergedBranch" + // DirtyWorktree: removing data refused because the worktree has changes. + DirtyWorktree = "DirtyWorktree" + // Internal: an unexpected error without a code of its own. + Internal = "Internal" +) diff --git a/internal/git/git.go b/internal/git/git.go new file mode 100644 index 0000000..a7076fb --- /dev/null +++ b/internal/git/git.go @@ -0,0 +1,147 @@ +// Package git answers what siding needs to know about a worktree by running the git CLI: where +// it is, which branch it is on, and whether removing its stack's data could lose work. +package git + +import ( + "bytes" + "context" + "errors" + "fmt" + "os/exec" + "path/filepath" + "strings" + + "github.com/tweinmann/siding/internal/fail" +) + +// Worktree is a git worktree. +type Worktree struct { + // Top is the absolute path of the worktree's top-level directory. + Top string + // CommonDir is the absolute path of the repository's common git directory, the same for + // every worktree of the repository. + CommonDir string + // Branch is the checked-out branch, without refs/heads/. + Branch string +} + +// RepoName is the name of the directory holding the common git directory: the main worktree's +// directory, the same for every worktree. +func (w Worktree) RepoName() string { + return filepath.Base(filepath.Dir(w.CommonDir)) +} + +// Client runs git. +type Client struct { + // Binary is the git executable; empty means "git" from PATH. + Binary string +} + +func (c Client) run(ctx context.Context, dir string, args ...string) (string, error) { + bin := c.Binary + if bin == "" { + bin = "git" + } + cmd := exec.CommandContext(ctx, bin, append([]string{"-C", dir}, args...)...) + var out, errOut bytes.Buffer + cmd.Stdout = &out + cmd.Stderr = &errOut + err := cmd.Run() + if err != nil { + var exit *exec.ExitError + if errors.As(err, &exit) { + return out.String(), &exitError{code: exit.ExitCode(), stderr: strings.TrimSpace(errOut.String())} + } + return "", fmt.Errorf("run git: %w", err) + } + return out.String(), nil +} + +type exitError struct { + code int + stderr string +} + +func (e *exitError) Error() string { return fmt.Sprintf("git exited %d: %s", e.code, e.stderr) } + +func exitCode(err error) int { + var e *exitError + if errors.As(err, &e) { + return e.code + } + return -1 +} + +// Worktree returns the worktree containing dir. It fails with NotAGitRepository outside a +// worktree and with NoBranch when HEAD is detached. +func (c Client) Worktree(ctx context.Context, dir string) (Worktree, error) { + out, err := c.run(ctx, dir, "rev-parse", "--path-format=absolute", "--show-toplevel", "--git-common-dir") + if err != nil { + if exitCode(err) > 0 { + return Worktree{}, fail.New(fail.NotAGitRepository, "%s is not inside a git worktree", dir). + WithHint("run siding in a git worktree of the repository whose stack you want") + } + return Worktree{}, err + } + lines := strings.Split(strings.TrimSpace(out), "\n") + if len(lines) != 2 { + return Worktree{}, fail.New(fail.NotAGitRepository, "%s is not inside a git worktree", dir) + } + w := Worktree{Top: lines[0], CommonDir: lines[1]} + branch, err := c.run(ctx, dir, "symbolic-ref", "--quiet", "--short", "HEAD") + if err != nil { + if exitCode(err) > 0 { + return Worktree{}, fail.New(fail.NoBranch, "HEAD of %s is detached", w.Top). + WithHint("check out a branch: every stack belongs to one branch") + } + return Worktree{}, err + } + w.Branch = strings.TrimSpace(branch) + return w, nil +} + +// Dirty reports whether git status lists any change in the worktree, untracked files +// included and ignored files excluded. +func (c Client) Dirty(ctx context.Context, top string) (bool, error) { + out, err := c.run(ctx, top, "status", "--porcelain") + if err != nil { + return false, err + } + return strings.TrimSpace(out) != "", nil +} + +// DefaultBranch returns the ref the repository merges into: what origin/HEAD points to, else +// a local main, else a local master. It returns "" when there is none. +func (c Client) DefaultBranch(ctx context.Context, top string) (string, error) { + if out, err := c.run(ctx, top, "symbolic-ref", "--quiet", "refs/remotes/origin/HEAD"); err == nil { + return strings.TrimSpace(out), nil + } else if exitCode(err) < 0 { + return "", err + } + for _, ref := range []string{"refs/heads/main", "refs/heads/master"} { + if _, err := c.run(ctx, top, "rev-parse", "--verify", "--quiet", ref); err == nil { + return ref, nil + } else if exitCode(err) < 0 { + return "", err + } + } + return "", nil +} + +// Merged reports whether the tip of branch is contained in the default branch, and which ref +// that is. Without a default branch it reports false. +func (c Client) Merged(ctx context.Context, top, branch string) (merged bool, base string, err error) { + base, err = c.DefaultBranch(ctx, top) + if err != nil || base == "" { + return false, "", err + } + _, err = c.run(ctx, top, "merge-base", "--is-ancestor", "refs/heads/"+branch, base) + switch code := exitCode(err); { + case err == nil: + return true, base, nil + case code == 1: + return false, base, nil + default: + return false, base, err + } +} diff --git a/internal/git/git_test.go b/internal/git/git_test.go new file mode 100644 index 0000000..d97d4dc --- /dev/null +++ b/internal/git/git_test.go @@ -0,0 +1,158 @@ +package git + +import ( + "context" + "errors" + "os" + "os/exec" + "path/filepath" + "testing" + + "github.com/tweinmann/siding/internal/fail" +) + +// sh runs git in dir with a fixed identity and without the developer's configuration. +func sh(t *testing.T, dir string, args ...string) { + t.Helper() + cmd := exec.Command("git", append([]string{"-C", dir}, args...)...) + cmd.Env = append(os.Environ(), + "GIT_CONFIG_GLOBAL=/dev/null", "GIT_CONFIG_NOSYSTEM=1", + "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@example.com", + "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@example.com") + if out, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("git %v: %v\n%s", args, err, out) + } +} + +// newRepo creates /shop with one commit on main and returns its path. +func newRepo(t *testing.T) string { + t.Helper() + dir := filepath.Join(t.TempDir(), "shop") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + sh(t, dir, "init", "-q", "-b", "main") + sh(t, dir, "commit", "-q", "--allow-empty", "-m", "first") + return dir +} + +func code(err error) string { + var e *fail.Error + if errors.As(err, &e) { + return e.Code + } + return "" +} + +func TestWorktree(t *testing.T) { + t.Parallel() + ctx := context.Background() + main := newRepo(t) + linked := filepath.Join(filepath.Dir(main), "shop-login") + sh(t, main, "worktree", "add", "-q", "-b", "feature/login", linked) + sub := filepath.Join(linked, "web", "src") + if err := os.MkdirAll(sub, 0o755); err != nil { + t.Fatal(err) + } + detached := filepath.Join(filepath.Dir(main), "shop-detached") + sh(t, main, "worktree", "add", "-q", "--detach", detached) + + tests := []struct { + name, dir, top, branch, code string + }{ + {name: "main worktree", dir: main, top: main, branch: "main"}, + {name: "linked worktree from a subdirectory", dir: sub, top: linked, branch: "feature/login"}, + {name: "detached", dir: detached, code: fail.NoBranch}, + {name: "outside", dir: t.TempDir(), code: fail.NotAGitRepository}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + w, err := Client{}.Worktree(ctx, tt.dir) + if tt.code != "" { + if code(err) != tt.code { + t.Fatalf("err %v, want code %s", err, tt.code) + } + return + } + if err != nil { + t.Fatal(err) + } + if w.Top != tt.top || w.Branch != tt.branch || w.RepoName() != "shop" { + t.Errorf("got %+v (repo %s), want top %s, branch %s, repo shop", w, w.RepoName(), tt.top, tt.branch) + } + }) + } +} + +func TestDirty(t *testing.T) { + t.Parallel() + ctx := context.Background() + dir := newRepo(t) + dirty, err := Client{}.Dirty(ctx, dir) + if err != nil || dirty { + t.Fatalf("fresh repository: dirty %v, err %v", dirty, err) + } + // .siding/ ignores itself, so siding's own files never make a worktree dirty. + if err := os.MkdirAll(filepath.Join(dir, ".siding"), 0o755); err != nil { + t.Fatal(err) + } + for name, content := range map[string]string{".siding/.gitignore": "*\n", ".siding/override.yml": "services: {}\n"} { + if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o644); err != nil { + t.Fatal(err) + } + } + if dirty, err := (Client{}).Dirty(ctx, dir); err != nil || dirty { + t.Fatalf("with .siding/: dirty %v, err %v", dirty, err) + } + if err := os.WriteFile(filepath.Join(dir, "new.txt"), []byte("x"), 0o644); err != nil { + t.Fatal(err) + } + if dirty, err := (Client{}).Dirty(ctx, dir); err != nil || !dirty { + t.Fatalf("with an untracked file: dirty %v, err %v", dirty, err) + } +} + +func TestMerged(t *testing.T) { + t.Parallel() + ctx := context.Background() + + t.Run("merged and unmerged against main", func(t *testing.T) { + t.Parallel() + dir := newRepo(t) + sh(t, dir, "branch", "done") + sh(t, dir, "switch", "-q", "-c", "wip") + sh(t, dir, "commit", "-q", "--allow-empty", "-m", "wip") + for branch, want := range map[string]bool{"done": true, "wip": false, "main": true} { + merged, base, err := Client{}.Merged(ctx, dir, branch) + if err != nil || merged != want || base != "refs/heads/main" { + t.Errorf("%s: merged %v, base %q, err %v; want %v against refs/heads/main", branch, merged, base, err, want) + } + } + }) + + t.Run("origin/HEAD wins", func(t *testing.T) { + t.Parallel() + origin := newRepo(t) + clone := filepath.Join(t.TempDir(), "clone") + sh(t, filepath.Dir(clone), "clone", "-q", origin, clone) + merged, base, err := Client{}.Merged(ctx, clone, "main") + if err != nil || !merged || base != "refs/remotes/origin/main" { + t.Errorf("merged %v, base %q, err %v; want true against refs/remotes/origin/main", merged, base, err) + } + }) + + t.Run("no default branch", func(t *testing.T) { + t.Parallel() + dir := filepath.Join(t.TempDir(), "r") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + sh(t, dir, "init", "-q", "-b", "trunk") + sh(t, dir, "commit", "-q", "--allow-empty", "-m", "first") + merged, base, err := Client{}.Merged(ctx, dir, "trunk") + if err != nil || merged || base != "" { + t.Errorf("merged %v, base %q, err %v; want unmerged without a base", merged, base, err) + } + }) +} diff --git a/internal/probe/probe.go b/internal/probe/probe.go new file mode 100644 index 0000000..53e021d --- /dev/null +++ b/internal/probe/probe.go @@ -0,0 +1,53 @@ +// Package probe asks the proxy whether a routed service answers, without DNS: the request goes +// to 127.0.0.1 on the proxy's port with the stack's host in the Host header. +package probe + +import ( + "context" + "io" + "net/http" + "time" +) + +// Response is what the proxy answered. +type Response struct { + Status int + // Service is the value of the X-Siding-Service header, set only on the service's own + // responses. + Service string +} + +// Answered reports whether the response came from service (/) itself and +// not from the proxy: it carries the service header and is not a gateway error. +func (r Response) Answered(service string) bool { + switch r.Status { + case http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout: + return false + } + return r.Service == service +} + +// HTTP probes over plain HTTP. +type HTTP struct { + // Header is the response header naming the service. + Header string +} + +// Get sends GET path with Host: host to addr (host:port) and follows no redirect. +func (p HTTP) Get(ctx context.Context, addr, host, path string) (Response, error) { + ctx, cancel := context.WithTimeout(ctx, 3*time.Second) + defer cancel() + req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://"+addr+path, nil) + if err != nil { + return Response{}, err + } + req.Host = host + c := http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }} + resp, err := c.Do(req) + if err != nil { + return Response{}, err + } + defer func() { _ = resp.Body.Close() }() + _, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 1<<16)) + return Response{Status: resp.StatusCode, Service: resp.Header.Get(p.Header)}, nil +} diff --git a/internal/probe/probe_test.go b/internal/probe/probe_test.go new file mode 100644 index 0000000..4e5ac6a --- /dev/null +++ b/internal/probe/probe_test.go @@ -0,0 +1,55 @@ +package probe + +import ( + "context" + "net/http" + "net/http/httptest" + "strings" + "testing" +) + +func TestGet(t *testing.T) { + t.Parallel() + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.Host != "shop.main.localhost" { + http.Error(w, "404 page not found", http.StatusNotFound) // like Traefik without a router + return + } + switch r.URL.Path { + case "/": + w.Header().Set("X-Siding-Service", "shop-main/web") + http.Redirect(w, r, "/login", http.StatusFound) + case "/api": + w.Header().Set("X-Siding-Service", "shop-main/api") + w.WriteHeader(http.StatusNotFound) // the app's own 404 + default: + w.Header().Set("X-Siding-Service", "shop-main/gone") + w.WriteHeader(http.StatusBadGateway) + } + })) + t.Cleanup(srv.Close) + addr := strings.TrimPrefix(srv.URL, "http://") + p := HTTP{Header: "X-Siding-Service"} + tests := []struct { + name, host, path, service string + answered bool + }{ + {name: "redirect counts", host: "shop.main.localhost", path: "/", service: "shop-main/web", answered: true}, + {name: "the app's 404 counts", host: "shop.main.localhost", path: "/api", service: "shop-main/api", answered: true}, + {name: "the proxy's 404 does not", host: "other.localhost", path: "/", service: "shop-main/web"}, + {name: "a gateway error does not", host: "shop.main.localhost", path: "/x", service: "shop-main/gone"}, + {name: "another service's answer does not", host: "shop.main.localhost", path: "/api", service: "shop-main/web"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + resp, err := p.Get(context.Background(), addr, tt.host, tt.path) + if err != nil { + t.Fatal(err) + } + if got := resp.Answered(tt.service); got != tt.answered { + t.Errorf("response %+v: answered %v, want %v", resp, got, tt.answered) + } + }) + } +} diff --git a/internal/proxy/proxy.go b/internal/proxy/proxy.go new file mode 100644 index 0000000..5c65ccd --- /dev/null +++ b/internal/proxy/proxy.go @@ -0,0 +1,66 @@ +// Package proxy describes siding-proxy: one Traefik with the Docker provider, run as its own +// Compose project, through which every stack is reached by host and path. +package proxy + +import ( + "fmt" + "strings" +) + +// Image is the Traefik image siding-proxy runs; the devcontainer's SIDING_TRAEFIK_IMAGE is the +// same. +const Image = "traefik:v3.7.16" + +// Config names the proxy and its network. Tests use their own, so they never touch a +// developer's siding-proxy. +type Config struct { + Project string // Compose project + Container string // container name + Network string // Docker network every routed service joins + Image string + DefaultPort int // host port on 127.0.0.1 when setup gets no --port +} + +// Default is the proxy of a real installation. +func Default() Config { + return Config{Project: "siding", Container: "siding-proxy", Network: "siding", Image: Image, DefaultPort: 80} +} + +// ComposeFile renders the proxy's compose file. The constraint keeps Traefik to containers +// siding labelled for this proxy's network, so foreign Traefik labels on the same daemon are +// never exposed. +func ComposeFile(c Config, port int, machine string) []byte { + constraint := fmt.Sprintf("LabelRegex(`dev.siding.repo`, `.+`) && Label(`traefik.docker.network`, `%s`)", c.Network) + var b strings.Builder + fmt.Fprintf(&b, `# Written by siding setup. Do not edit: siding setup rewrites it. +services: + traefik: + image: %q + container_name: %q + restart: unless-stopped + command: + - --entrypoints.web.address=:80 + - --providers.docker=true + - --providers.docker.exposedbydefault=false + - --providers.docker.network=%s + - %q + - --ping=true + ports: + - "127.0.0.1:%d:80" + volumes: + - /var/run/docker.sock:/var/run/docker.sock:ro + networks: [proxy] + labels: + dev.siding.machine: %q + healthcheck: + test: ["CMD", "traefik", "healthcheck", "--ping"] + interval: 2s + timeout: 2s + retries: 30 +networks: + proxy: + name: %q + external: true +`, c.Image, c.Container, c.Network, "--providers.docker.constraints="+constraint, port, machine, c.Network) + return []byte(b.String()) +} diff --git a/internal/stack/identity.go b/internal/stack/identity.go new file mode 100644 index 0000000..7e7f9a3 --- /dev/null +++ b/internal/stack/identity.go @@ -0,0 +1,111 @@ +// Package stack derives everything about a stack that follows from its worktree and its +// compose file without asking Docker: names, labels, routes and the override siding writes. +// Everything here is a pure function. +package stack + +import ( + "crypto/sha256" + "encoding/hex" + "strings" +) + +// Labels siding puts on what it creates. The conventions in CLAUDE.md list them. +const ( + LabelRepo = "dev.siding.repo" + LabelBranch = "dev.siding.branch" + LabelWorktree = "dev.siding.worktree" + LabelMachine = "dev.siding.machine" + LabelAgent = "dev.siding.agent" + LabelRoute = "dev.siding.route" + LabelPort = "dev.siding.port" + + // LabelProject is Compose's own label naming a container's project. + LabelProject = "com.docker.compose.project" + // LabelService is Compose's own label naming a container's service. + LabelService = "com.docker.compose.service" +) + +// maxSlug is the longest slug; longer names are cut and get a hash suffix. +const maxSlug = 40 + +// Slug turns a name into lowercase letters, digits and single dashes, at most 40 characters. +func Slug(name string) string { + var b strings.Builder + dash := false + for _, r := range strings.ToLower(name) { + if (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9') { + if dash && b.Len() > 0 { + b.WriteByte('-') + } + dash = false + b.WriteRune(r) + continue + } + dash = true + } + s := b.String() + if s == "" || len(s) > maxSlug { + sum := sha256.Sum256([]byte(name)) + suffix := hex.EncodeToString(sum[:])[:6] + if s == "" { + return "x-" + suffix + } + s = strings.TrimRight(s[:maxSlug-7], "-") + "-" + suffix + } + return s +} + +// Identity names one stack. +type Identity struct { + Repo string // repository name, e.g. shop + Branch string // branch name, e.g. feature/login + Worktree string // absolute path of the worktree + Machine string // stable machine id + Agent string // optional agent name +} + +// RepoSlug is the slug of the repository name. +func (id Identity) RepoSlug() string { return Slug(id.Repo) } + +// BranchSlug is the slug of the branch. +func (id Identity) BranchSlug() string { return Slug(id.Branch) } + +// Project is the Compose project, -. +func (id Identity) Project() string { return id.RepoSlug() + "-" + id.BranchSlug() } + +// Host is the stack's host, ..localhost. +func (id Identity) Host() string { return id.RepoSlug() + "." + id.BranchSlug() + ".localhost" } + +// ServiceHost is the host of one service, ...localhost. +func (id Identity) ServiceHost(service string) string { return Slug(service) + "." + id.Host() } + +// Labels are the labels of the stack's networks and volumes. They never change between two +// ups of one project: Compose recreates a network whose labels changed. +func (id Identity) Labels() map[string]string { + return map[string]string{ + LabelRepo: id.Repo, + LabelBranch: id.Branch, + LabelWorktree: id.Worktree, + LabelMachine: id.Machine, + } +} + +// ContainerLabels are the labels of the stack's containers: Labels plus the agent, if any. +func (id Identity) ContainerLabels() map[string]string { + l := id.Labels() + if id.Agent != "" { + l[LabelAgent] = id.Agent + } + return l +} + +// FromLabels reads an Identity back from a container's labels. +func FromLabels(l map[string]string) Identity { + return Identity{ + Repo: l[LabelRepo], + Branch: l[LabelBranch], + Worktree: l[LabelWorktree], + Machine: l[LabelMachine], + Agent: l[LabelAgent], + } +} diff --git a/internal/stack/identity_test.go b/internal/stack/identity_test.go new file mode 100644 index 0000000..20d6f10 --- /dev/null +++ b/internal/stack/identity_test.go @@ -0,0 +1,126 @@ +package stack + +import ( + "os" + "path/filepath" + "regexp" + "strings" + "testing" +) + +func TestSlug(t *testing.T) { + t.Parallel() + tests := []struct { + name, in, want string + }{ + {name: "plain", in: "main", want: "main"}, + {name: "slash and case", in: "feature/Login_Form", want: "feature-login-form"}, + {name: "runs of symbols", in: "a//b__c..d", want: "a-b-c-d"}, + {name: "leading and trailing", in: "-/x/-", want: "x"}, + {name: "non-ASCII", in: "fix/über", want: "fix-ber"}, + {name: "40 characters stay", in: strings.Repeat("a", 40), want: strings.Repeat("a", 40)}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + if got := Slug(tt.in); got != tt.want { + t.Errorf("Slug(%q) = %q, want %q", tt.in, got, tt.want) + } + }) + } + + hashed := []struct { + name, in string + shape *regexp.Regexp + }{ + {name: "41 characters are cut", in: strings.Repeat("a", 41), shape: regexp.MustCompile(`^a{33}-[0-9a-f]{6}$`)}, + {name: "60 characters are cut", in: strings.Repeat("abcdefghij", 6), shape: regexp.MustCompile(`^(abcdefghij){3}abc-[0-9a-f]{6}$`)}, + {name: "only symbols", in: "///", shape: regexp.MustCompile(`^x-[0-9a-f]{6}$`)}, + } + for _, tt := range hashed { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + if got := Slug(tt.in); !tt.shape.MatchString(got) { + t.Errorf("Slug(%q) = %q, want it to match %s", tt.in, got, tt.shape) + } + }) + } + + t.Run("long names with the same start differ", func(t *testing.T) { + t.Parallel() + long := strings.Repeat("abcdefghij", 6) + if a, b := Slug(long), Slug(long[:59]+"z"); a == b { + t.Errorf("both slugs are %q", a) + } + }) +} + +func TestIdentity(t *testing.T) { + t.Parallel() + id := Identity{Repo: "shop", Branch: "feature/login", Worktree: "/src/shop-login", Machine: "m1"} + if got := id.Project(); got != "shop-feature-login" { + t.Errorf("Project = %q", got) + } + if got := id.Host(); got != "shop.feature-login.localhost" { + t.Errorf("Host = %q", got) + } + if got := id.ServiceHost("Mail_Hog"); got != "mail-hog.shop.feature-login.localhost" { + t.Errorf("ServiceHost = %q", got) + } + if _, ok := id.ContainerLabels()[LabelAgent]; ok { + t.Error("an agent label without an agent") + } + id.Agent = "claude" + if got := id.ContainerLabels()[LabelAgent]; got != "claude" { + t.Errorf("agent label = %q", got) + } + if _, ok := id.Labels()[LabelAgent]; ok { + t.Error("networks and volumes must not carry the agent label") + } + if got := FromLabels(id.ContainerLabels()); got != id { + t.Errorf("FromLabels = %+v, want %+v", got, id) + } + want := map[string]string{LabelRepo: "shop", LabelBranch: "feature/login", LabelWorktree: "/src/shop-login", LabelMachine: "m1"} + for k, v := range want { + if id.Labels()[k] != v { + t.Errorf("label %s = %q, want %q", k, id.Labels()[k], v) + } + } +} + +func TestMachineID(t *testing.T) { + t.Parallel() + + t.Run("from the operating system", func(t *testing.T) { + t.Parallel() + home := t.TempDir() + a, err := MachineID("4c4c4544-not-a-real-id\n", home) + if err != nil { + t.Fatal(err) + } + b, _ := MachineID("4c4c4544-not-a-real-id", home) + if a != b || len(a) != 16 || strings.Contains(a, "4c4c4544") { + t.Errorf("ids %q and %q: want the same 16 hex digits, without the raw id", a, b) + } + if _, err := os.Stat(filepath.Join(home, ".siding")); err == nil { + t.Error("wrote ~/.siding although the operating system has an id") + } + }) + + t.Run("generated once", func(t *testing.T) { + t.Parallel() + home := t.TempDir() + a, err := MachineID("", home) + if err != nil { + t.Fatal(err) + } + b, err := MachineID("", home) + if err != nil || a != b { + t.Fatalf("second call: %q, %v; want %q", b, err, a) + } + saved, _ := os.ReadFile(filepath.Join(home, ".siding", "machine-id")) + if strings.TrimSpace(string(saved)) != a { + t.Errorf("~/.siding/machine-id holds %q, want %q", saved, a) + } + }) +} diff --git a/internal/stack/machineid.go b/internal/stack/machineid.go new file mode 100644 index 0000000..c4ca583 --- /dev/null +++ b/internal/stack/machineid.go @@ -0,0 +1,68 @@ +package stack + +import ( + "crypto/rand" + "crypto/sha256" + "encoding/hex" + "errors" + "fmt" + "io/fs" + "os" + "os/exec" + "path/filepath" + "regexp" + "runtime" + "strings" +) + +// MachineID returns the stable id for dev.siding.machine: a one-way hash of the operating +// system's machine id, so the raw id never appears in a label. Without one it uses, and on +// first use writes, a random id in /.siding/machine-id. +func MachineID(osID, home string) (string, error) { + if osID = strings.TrimSpace(osID); osID != "" { + sum := sha256.Sum256([]byte("siding-machine-id:" + osID)) + return hex.EncodeToString(sum[:])[:16], nil + } + path := filepath.Join(home, ".siding", "machine-id") + if b, err := os.ReadFile(path); err == nil { + if id := strings.TrimSpace(string(b)); id != "" { + return id, nil + } + } else if !errors.Is(err, fs.ErrNotExist) { + return "", fmt.Errorf("read machine id: %w", err) + } + var raw [8]byte + if _, err := rand.Read(raw[:]); err != nil { + return "", err + } + id := hex.EncodeToString(raw[:]) + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return "", fmt.Errorf("write machine id: %w", err) + } + if err := os.WriteFile(path, []byte(id+"\n"), 0o644); err != nil { + return "", fmt.Errorf("write machine id: %w", err) + } + return id, nil +} + +var platformUUID = regexp.MustCompile(`"IOPlatformUUID" = "([^"]+)"`) + +// OSMachineID returns the operating system's machine id: /etc/machine-id on Linux, the +// platform UUID on macOS, or "" when there is none. +func OSMachineID() string { + if runtime.GOOS == "darwin" { + out, err := exec.Command("ioreg", "-rd1", "-c", "IOPlatformExpertDevice").Output() + if err != nil { + return "" + } + if m := platformUUID.FindSubmatch(out); m != nil { + return string(m[1]) + } + return "" + } + b, err := os.ReadFile("/etc/machine-id") + if err != nil { + return "" + } + return strings.TrimSpace(string(b)) +} diff --git a/internal/stack/override.go b/internal/stack/override.go new file mode 100644 index 0000000..2c56794 --- /dev/null +++ b/internal/stack/override.go @@ -0,0 +1,117 @@ +package stack + +import ( + "bytes" + "encoding/json" + "fmt" + "maps" + "slices" + "strings" + + "github.com/tweinmann/siding/internal/compose" +) + +// OverrideFile is where siding writes the override, relative to the worktree. +const OverrideFile = ".siding/override.yml" + +// ProxyNetworkKey is the key of the proxy network in the override. It differs from every +// network name a user is likely to pick, so the override never redefines one of theirs. +const ProxyNetworkKey = "siding_proxy" + +// ServiceHeader is the response header every siding router adds, naming /. +// A response that carries it came from the service, not from the proxy (design D9). +const ServiceHeader = "X-Siding-Service" + +// Override renders .siding/override.yml: no published ports, siding's labels on every +// container, network and volume of the project, and for every route the proxy network and +// the Traefik labels. network is the proxy's Docker network. +func Override(p *compose.Project, id Identity, routes []Route, network string) []byte { + var b strings.Builder + project := id.Project() + fmt.Fprintf(&b, "# Written by siding for the Compose project %s. Do not edit: siding up rewrites it.\n", project) + b.WriteString("services:\n") + for _, name := range p.ServiceNames() { + s := p.Services[name] + fmt.Fprintf(&b, " %s:\n", q(name)) + b.WriteString(" ports: !reset []\n") + labels := id.ContainerLabels() + if r, ok := Find(routes, name); ok { + b.WriteString(" networks:\n") + for _, n := range s.Networks { + fmt.Fprintf(&b, " %s: {}\n", q(n)) + } + fmt.Fprintf(&b, " %s: {}\n", ProxyNetworkKey) + maps.Copy(labels, traefikLabels(id, r, network)) + } + writeLabels(&b, " ", labels) + } + + b.WriteString("networks:\n") + for _, key := range sortedKeys(p.Networks) { + if p.Networks[key].External { + continue + } + fmt.Fprintf(&b, " %s:\n", q(key)) + writeLabels(&b, " ", id.Labels()) + } + fmt.Fprintf(&b, " %s:\n name: %s\n external: true\n", ProxyNetworkKey, q(network)) + + volumes := slices.DeleteFunc(sortedKeys(p.Volumes), func(k string) bool { return p.Volumes[k].External }) + if len(volumes) > 0 { + b.WriteString("volumes:\n") + for _, key := range volumes { + fmt.Fprintf(&b, " %s:\n", q(key)) + writeLabels(&b, " ", id.Labels()) + } + } + return []byte(b.String()) +} + +// traefikLabels route the service host to the service and, for a route path, the path on the +// stack's host. Router names are unique per daemon because they start with the project. +func traefikLabels(id Identity, r Route, network string) map[string]string { + name := id.Project() + "-" + Slug(r.Service) + middleware := name + "-id" + l := map[string]string{ + "traefik.enable": "true", + "traefik.docker.network": network, + "traefik.http.services." + name + ".loadbalancer.server.port": fmt.Sprint(r.Port), + "traefik.http.middlewares." + middleware + ".headers.customresponseheaders." + ServiceHeader: id.Project() + "/" + r.Service, + } + router := func(router, rule string) { + prefix := "traefik.http.routers." + router + l[prefix+".rule"] = rule + l[prefix+".entrypoints"] = "web" + l[prefix+".service"] = name + l[prefix+".middlewares"] = middleware + } + router(name, "Host(`"+id.ServiceHost(r.Service)+"`)") + switch r.Path { + case "": + case "/": + router(name+"-route", "Host(`"+id.Host()+"`)") + default: + router(name+"-route", "Host(`"+id.Host()+"`) && PathPrefix(`"+r.Path+"`)") + } + return l +} + +func writeLabels(b *strings.Builder, indent string, labels map[string]string) { + b.WriteString(indent + "labels:\n") + for _, k := range sortedKeys(labels) { + fmt.Fprintf(b, "%s %s: %s\n", indent, q(k), q(labels[k])) + } +} + +// q quotes a YAML scalar. A JSON string is a valid YAML double-quoted scalar. +func q(s string) string { + var b bytes.Buffer + enc := json.NewEncoder(&b) + enc.SetEscapeHTML(false) + _ = enc.Encode(s) + return strings.TrimSuffix(b.String(), "\n") +} + +func sortedKeys[V any](m map[string]V) []string { + return slices.Sorted(maps.Keys(m)) +} diff --git a/internal/stack/redact.go b/internal/stack/redact.go new file mode 100644 index 0000000..c0eda37 --- /dev/null +++ b/internal/stack/redact.go @@ -0,0 +1,90 @@ +package stack + +import ( + "bufio" + "slices" + "strconv" + "strings" +) + +// minSecret is the shortest value siding redacts; shorter ones would wreck every line. +const minSecret = 4 + +// Redactor replaces known secret values in log lines with ***. +type Redactor struct { + values []string // longest first +} + +// NewRedactor redacts the given values except short ones, numbers and booleans. +func NewRedactor(values ...string) *Redactor { + seen := map[string]bool{} + r := &Redactor{} + for _, v := range values { + if len(v) < minSecret || seen[v] || harmless(v) { + continue + } + seen[v] = true + r.values = append(r.values, v) + } + slices.SortFunc(r.values, func(a, b string) int { return len(b) - len(a) }) + return r +} + +func harmless(v string) bool { + switch strings.ToLower(v) { + case "true", "false", "yes", "no", "on", "off": + return true + } + _, err := strconv.ParseFloat(v, 64) + return err == nil +} + +// Line returns line with every known value replaced by ***. +func (r *Redactor) Line(line string) string { + for _, v := range r.values { + line = strings.ReplaceAll(line, v, "***") + } + return line +} + +// Lines redacts every line. +func (r *Redactor) Lines(lines []string) []string { + out := make([]string, len(lines)) + for i, l := range lines { + out[i] = r.Line(l) + } + return out +} + +// DotEnvValues returns the values a .env file defines, read with Compose's rules: KEY=VALUE +// per line, an optional "export ", comments starting with #, single or double quotes. +func DotEnvValues(content string) []string { + var values []string + sc := bufio.NewScanner(strings.NewReader(content)) + for sc.Scan() { + line := strings.TrimSpace(sc.Text()) + if line == "" || strings.HasPrefix(line, "#") { + continue + } + line = strings.TrimPrefix(line, "export ") + _, value, ok := strings.Cut(line, "=") + if !ok { + continue + } + value = strings.TrimSpace(value) + switch { + case len(value) >= 2 && (value[0] == '"' || value[0] == '\''): + if end := strings.IndexByte(value[1:], value[0]); end >= 0 { + value = value[1 : end+1] + } + default: + if i := strings.Index(value, " #"); i >= 0 { + value = strings.TrimSpace(value[:i]) + } + } + if value != "" { + values = append(values, value) + } + } + return values +} diff --git a/internal/stack/redact_test.go b/internal/stack/redact_test.go new file mode 100644 index 0000000..bc277aa --- /dev/null +++ b/internal/stack/redact_test.go @@ -0,0 +1,38 @@ +package stack + +import ( + "reflect" + "testing" +) + +func TestDotEnvValues(t *testing.T) { + t.Parallel() + content := `# a comment +DB_PASSWORD=not-a-real-token +export API_KEY="not-a-real-key" # trailing comment +SINGLE='it''s' +PLAIN=value with spaces # comment +EMPTY= +NO_EQUALS +` + want := []string{"not-a-real-token", "not-a-real-key", "it", "value with spaces"} + if got := DotEnvValues(content); !reflect.DeepEqual(got, want) { + t.Errorf("got %q, want %q", got, want) + } +} + +func TestRedactor(t *testing.T) { + t.Parallel() + r := NewRedactor("not-a-real-token", "not-a-real", "on", "true", "3000", "abc", "development") + tests := []struct{ in, want string }{ + {in: "connecting with not-a-real-token", want: "connecting with ***"}, + {in: "listening on 3000 in development, debug true", want: "listening on 3000 in ***, debug true"}, + {in: "prefix not-a-real only", want: "prefix *** only"}, + {in: "abc stays, it is short", want: "abc stays, it is short"}, + } + for _, tt := range tests { + if got := r.Line(tt.in); got != tt.want { + t.Errorf("Line(%q) = %q, want %q", tt.in, got, tt.want) + } + } +} diff --git a/internal/stack/routing.go b/internal/stack/routing.go new file mode 100644 index 0000000..4d0d9a4 --- /dev/null +++ b/internal/stack/routing.go @@ -0,0 +1,116 @@ +package stack + +import ( + "fmt" + "regexp" + "slices" + "strconv" + "strings" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" +) + +// Route says how the proxy reaches one HTTP service. +type Route struct { + Service string + // Port is the container port the proxy sends requests to. + Port int + // Path is the service's path on the stack's host ("/" for the root), or "" when it has + // only its service host. + Path string +} + +// validPath keeps route paths to characters that are safe inside a Traefik rule. +var validPath = regexp.MustCompile(`^/[A-Za-z0-9._~/-]*$`) + +// Routes derives the routing table of a project: one Route per HTTP service that is not +// hidden, sorted by service. Warnings name services siding cannot route. +func Routes(p *compose.Project) (routes []Route, warnings []string, err error) { + paths := map[string]string{} // path -> service + for _, name := range p.ServiceNames() { + s := p.Services[name] + label, hasLabel := s.Labels[LabelRoute] + label = strings.TrimSpace(label) + if label == "false" { + continue + } + path := "" + if hasLabel && label != "" { + if !validPath.MatchString(label) { + return nil, nil, fail.New(fail.InvalidRoute, "service %s: %s must be \"false\" or a path such as \"/\" or \"/api\", got %q", name, LabelRoute, label) + } + path = label + if path != "/" { + path = strings.TrimRight(path, "/") + } + } + + port, ports, err := httpPort(name, s) + if err != nil { + return nil, nil, err + } + if s.NetworkMode != "" { + if path != "" { + return nil, nil, fail.New(fail.InvalidRoute, "service %s has %s %q but uses network_mode %q, so the proxy cannot reach it", name, LabelRoute, label, s.NetworkMode). + WithHint("remove network_mode or the route label") + } + if port != 0 || len(ports) > 1 { + warnings = append(warnings, fmt.Sprintf("service %s uses network_mode %q and is not routed", name, s.NetworkMode)) + } + continue + } + if path != "" && port == 0 { + if len(ports) > 1 { + return nil, nil, fail.New(fail.AmbiguousPort, "service %s has %s %q and the container ports %s", name, LabelRoute, label, joinInts(ports)). + WithHint("name the HTTP port with the label %s", LabelPort) + } + return nil, nil, fail.New(fail.InvalidRoute, "service %s has %s %q but no container port", name, LabelRoute, label). + WithHint("add expose: [\"\"] or the label %s to the service", LabelPort) + } + if port == 0 { + continue + } + if path != "" { + if other, taken := paths[path]; taken { + return nil, nil, fail.New(fail.InvalidRoute, "services %s and %s both have %s %q", other, name, LabelRoute, path) + } + paths[path] = name + } + routes = append(routes, Route{Service: name, Port: port, Path: path}) + } + return routes, warnings, nil +} + +// httpPort returns the HTTP port of a service, or 0 when it has none or several (returned as +// ports, for the error message). +func httpPort(name string, s compose.Service) (port int, ports []int, err error) { + if v, ok := s.Labels[LabelPort]; ok { + n, err := strconv.Atoi(strings.TrimSpace(v)) + if err != nil || n < 1 || n > 65535 { + return 0, nil, fail.New(fail.InvalidRoute, "service %s: %s must be a port number, got %q", name, LabelPort, v) + } + return n, []int{n}, nil + } + if len(s.Ports) == 1 { + return s.Ports[0], s.Ports, nil + } + return 0, s.Ports, nil +} + +func joinInts(ns []int) string { + s := make([]string, len(ns)) + for i, n := range ns { + s[i] = strconv.Itoa(n) + } + return strings.Join(s, ", ") +} + +// Find returns the route of a service. +func Find(routes []Route, service string) (Route, bool) { + i := slices.IndexFunc(routes, func(r Route) bool { return r.Service == service }) + if i < 0 { + return Route{}, false + } + return routes[i], true +} diff --git a/internal/stack/routing_test.go b/internal/stack/routing_test.go new file mode 100644 index 0000000..21a98b3 --- /dev/null +++ b/internal/stack/routing_test.go @@ -0,0 +1,193 @@ +package stack + +import ( + "errors" + "flag" + "os" + "os/exec" + "path/filepath" + "reflect" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/testutil" +) + +var withCompose = flag.Bool("compose", false, "also check that docker compose accepts every golden override") + +func svc(ports []int, labels map[string]string) compose.Service { + if labels == nil { + labels = map[string]string{} + } + return compose.Service{Ports: ports, Labels: labels, Networks: []string{"default"}} +} + +func TestRoutes(t *testing.T) { + t.Parallel() + tests := []struct { + name string + services map[string]compose.Service + routes []Route + warnings int + code string + }{ + { + name: "root and api", + services: map[string]compose.Service{ + "web": svc([]int{3000}, map[string]string{LabelRoute: "/"}), + "api": svc([]int{8080}, map[string]string{LabelRoute: "/api/"}), + }, + routes: []Route{{Service: "api", Port: 8080, Path: "/api"}, {Service: "web", Port: 3000, Path: "/"}}, + }, + { + name: "one port without label gets a service host", + services: map[string]compose.Service{"mail": svc([]int{8025}, nil)}, + routes: []Route{{Service: "mail", Port: 8025}}, + }, + { + name: "several ports without label are not routed", + services: map[string]compose.Service{"debug": svc([]int{3000, 9229}, nil)}, + }, + { + name: "several ports with the port label", + services: map[string]compose.Service{"web": svc([]int{3000, 9229}, map[string]string{LabelPort: "3000", LabelRoute: "/"})}, + routes: []Route{{Service: "web", Port: 3000, Path: "/"}}, + }, + { + name: "port label without ports", + services: map[string]compose.Service{"web": svc(nil, map[string]string{LabelPort: "80"})}, + routes: []Route{{Service: "web", Port: 80}}, + }, + { + name: "no port", + services: map[string]compose.Service{"worker": svc(nil, nil)}, + }, + { + name: "hidden", + services: map[string]compose.Service{"admin": svc([]int{80}, map[string]string{LabelRoute: "false"})}, + }, + { + name: "two services on one path", + services: map[string]compose.Service{"web": svc([]int{80}, map[string]string{LabelRoute: "/"}), "admin": svc([]int{80}, map[string]string{LabelRoute: "/"})}, + code: fail.InvalidRoute, + }, + { + name: "a route without a port", + services: map[string]compose.Service{"api": svc(nil, map[string]string{LabelRoute: "/api"})}, + code: fail.InvalidRoute, + }, + { + name: "a route with several ports", + services: map[string]compose.Service{"web": svc([]int{3000, 9229}, map[string]string{LabelRoute: "/"})}, + code: fail.AmbiguousPort, + }, + { + name: "not a path", + services: map[string]compose.Service{"web": svc([]int{80}, map[string]string{LabelRoute: "api"})}, + code: fail.InvalidRoute, + }, + { + name: "a path that would break the rule", + services: map[string]compose.Service{"web": svc([]int{80}, map[string]string{LabelRoute: "/a`) || Host(`x"})}, + code: fail.InvalidRoute, + }, + { + name: "a bad port label", + services: map[string]compose.Service{"web": svc([]int{80}, map[string]string{LabelPort: "http"})}, + code: fail.InvalidRoute, + }, + { + name: "network_mode without route warns", + services: map[string]compose.Service{"tools": {Ports: []int{7000}, Labels: map[string]string{}, NetworkMode: "host"}}, + warnings: 1, + }, + { + name: "network_mode with route fails", + services: map[string]compose.Service{"tools": {Ports: []int{7000}, Labels: map[string]string{LabelRoute: "/"}, NetworkMode: "host"}}, + code: fail.InvalidRoute, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + routes, warnings, err := Routes(&compose.Project{Services: tt.services}) + if tt.code != "" { + var e *fail.Error + if !errors.As(err, &e) || e.Code != tt.code { + t.Fatalf("err %v, want code %s", err, tt.code) + } + return + } + if err != nil { + t.Fatal(err) + } + if !reflect.DeepEqual(routes, tt.routes) || len(warnings) != tt.warnings { + t.Errorf("routes %+v, warnings %q; want %+v and %d warnings", routes, warnings, tt.routes, tt.warnings) + } + }) + } +} + +// Each directory in testdata/override holds a captured compose config (config.json, from +// compose.yaml next to it) and the override siding writes for it (override.yml). +func TestOverride(t *testing.T) { + t.Parallel() + id := Identity{Repo: "shop", Branch: "main", Worktree: "/src/shop", Machine: "0123456789abcdef", Agent: "claude"} + for _, dir := range []string{"routing"} { + t.Run(dir, func(t *testing.T) { + t.Parallel() + src := filepath.Join("..", "compose", "testdata", dir) + data, err := os.ReadFile(filepath.Join(src, "config.json")) + if err != nil { + t.Fatal(err) + } + p, err := compose.ParseConfig(data) + if err != nil { + t.Fatal(err) + } + routes, _, err := Routes(p) + if err != nil { + t.Fatal(err) + } + got := Override(p, id, routes, "siding") + golden := filepath.Join("testdata", "override", dir+".yml") + testutil.Golden(t, golden, got) + if *withCompose { + checkWithCompose(t, src, golden) + } + }) + } +} + +// checkWithCompose proves that Compose accepts the override and that the result has no +// published port and keeps the user's networks. +func checkWithCompose(t *testing.T, src, override string) { + t.Helper() + abs, _ := filepath.Abs(override) + cmd := exec.Command("docker", "compose", "-p", "shop-main", "-f", "compose.yaml", "-f", abs, "config", "--format", "json") + cmd.Dir = src + out, err := cmd.Output() + if err != nil { + t.Fatalf("docker compose config with the override: %v", err) + } + merged, err := compose.ParseConfig(out) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(out), `"published"`) { + t.Error("a port is still published") + } + for name, s := range merged.Services { + if s.Labels[LabelRepo] != "shop" { + t.Errorf("%s lacks the repo label", name) + } + } + if got := merged.Services["api"].Networks; !reflect.DeepEqual(got, []string{"backend", "default", ProxyNetworkKey}) { + t.Errorf("api networks %v", got) + } + if got := merged.Services["worker"].Networks; !reflect.DeepEqual(got, []string{"default"}) { + t.Errorf("worker networks %v", got) + } +} diff --git a/internal/stack/testdata/override/routing.yml b/internal/stack/testdata/override/routing.yml new file mode 100644 index 0000000..9c8b29e --- /dev/null +++ b/internal/stack/testdata/override/routing.yml @@ -0,0 +1,134 @@ +# Written by siding for the Compose project shop-main. Do not edit: siding up rewrites it. +services: + "admin": + ports: !reset [] + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "api": + ports: !reset [] + networks: + "backend": {} + "default": {} + siding_proxy: {} + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "traefik.docker.network": "siding" + "traefik.enable": "true" + "traefik.http.middlewares.shop-main-api-id.headers.customresponseheaders.X-Siding-Service": "shop-main/api" + "traefik.http.routers.shop-main-api-route.entrypoints": "web" + "traefik.http.routers.shop-main-api-route.middlewares": "shop-main-api-id" + "traefik.http.routers.shop-main-api-route.rule": "Host(`shop.main.localhost`) && PathPrefix(`/api`)" + "traefik.http.routers.shop-main-api-route.service": "shop-main-api" + "traefik.http.routers.shop-main-api.entrypoints": "web" + "traefik.http.routers.shop-main-api.middlewares": "shop-main-api-id" + "traefik.http.routers.shop-main-api.rule": "Host(`api.shop.main.localhost`)" + "traefik.http.routers.shop-main-api.service": "shop-main-api" + "traefik.http.services.shop-main-api.loadbalancer.server.port": "8080" + "db": + ports: !reset [] + networks: + "backend": {} + siding_proxy: {} + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "traefik.docker.network": "siding" + "traefik.enable": "true" + "traefik.http.middlewares.shop-main-db-id.headers.customresponseheaders.X-Siding-Service": "shop-main/db" + "traefik.http.routers.shop-main-db.entrypoints": "web" + "traefik.http.routers.shop-main-db.middlewares": "shop-main-db-id" + "traefik.http.routers.shop-main-db.rule": "Host(`db.shop.main.localhost`)" + "traefik.http.routers.shop-main-db.service": "shop-main-db" + "traefik.http.services.shop-main-db.loadbalancer.server.port": "5432" + "mail": + ports: !reset [] + networks: + "default": {} + siding_proxy: {} + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "traefik.docker.network": "siding" + "traefik.enable": "true" + "traefik.http.middlewares.shop-main-mail-id.headers.customresponseheaders.X-Siding-Service": "shop-main/mail" + "traefik.http.routers.shop-main-mail.entrypoints": "web" + "traefik.http.routers.shop-main-mail.middlewares": "shop-main-mail-id" + "traefik.http.routers.shop-main-mail.rule": "Host(`mail.shop.main.localhost`)" + "traefik.http.routers.shop-main-mail.service": "shop-main-mail" + "traefik.http.services.shop-main-mail.loadbalancer.server.port": "8025" + "tools": + ports: !reset [] + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "web": + ports: !reset [] + networks: + "default": {} + siding_proxy: {} + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "traefik.docker.network": "siding" + "traefik.enable": "true" + "traefik.http.middlewares.shop-main-web-id.headers.customresponseheaders.X-Siding-Service": "shop-main/web" + "traefik.http.routers.shop-main-web-route.entrypoints": "web" + "traefik.http.routers.shop-main-web-route.middlewares": "shop-main-web-id" + "traefik.http.routers.shop-main-web-route.rule": "Host(`shop.main.localhost`)" + "traefik.http.routers.shop-main-web-route.service": "shop-main-web" + "traefik.http.routers.shop-main-web.entrypoints": "web" + "traefik.http.routers.shop-main-web.middlewares": "shop-main-web-id" + "traefik.http.routers.shop-main-web.rule": "Host(`web.shop.main.localhost`)" + "traefik.http.routers.shop-main-web.service": "shop-main-web" + "traefik.http.services.shop-main-web.loadbalancer.server.port": "3000" + "worker": + ports: !reset [] + labels: + "dev.siding.agent": "claude" + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" +networks: + "backend": + labels: + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + "default": + labels: + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" + siding_proxy: + name: "siding" + external: true +volumes: + "data": + labels: + "dev.siding.branch": "main" + "dev.siding.machine": "0123456789abcdef" + "dev.siding.repo": "shop" + "dev.siding.worktree": "/src/shop" diff --git a/internal/verdict/verdict.go b/internal/verdict/verdict.go new file mode 100644 index 0000000..346fc6a --- /dev/null +++ b/internal/verdict/verdict.go @@ -0,0 +1,191 @@ +// Package verdict decides whether a stack runs: Ready only when every service is up, healthy +// where it has a healthcheck, and answering through the proxy where it has a route path. +// Evaluate is a pure function; status and up --wait call it once per poll. +package verdict + +import ( + "fmt" + "time" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/stack" +) + +// Verdicts. +const ( + Ready = "Ready" + Starting = "Starting" + Failed = "Failed" + TimedOut = "TimedOut" +) + +// Reasons a service is not ready. Part of the --json contract. +const ( + Exited = "Exited" + Unhealthy = "Unhealthy" + Restarting = "Restarting" + Missing = "Missing" + NotAnswering = "NotAnswering" + NotReady = "NotReady" +) + +// restartLimit is how many restarts while waiting count as a crash loop. +const restartLimit = 3 + +// Route is a routed service as the probe sees it. +type Route struct { + URL string + Port int +} + +// Input is everything one evaluation looks at. +type Input struct { + // Services are the compose file's services, in compose order. + Services []string + Containers []docker.Container + // Routes are the services with a route path. + Routes map[string]Route + // Answering holds, per routed service, whether the last probe got its own answer. + Answering map[string]bool + // RestartBase is each container's restart count when waiting began, by container ID. + RestartBase map[string]int + // Final is set when the time is up: what is not ready now is TimedOut. + Final bool + Timeout time.Duration +} + +// Service is the state of one service. +type Service struct { + Name string + State string // container state, or "missing" + Health string + // Answering is set for routed services only. + Answering *bool + Ready bool +} + +// Failure names the service that keeps the stack from being Ready. +type Failure struct { + Service string + Reason string + Hint string +} + +// Result is a verdict with the state of every service. +type Result struct { + Verdict string + Services []Service + Failure *Failure +} + +type outcome int + +const ( + ready outcome = iota + pending + failed +) + +// Evaluate returns the verdict for in. +func Evaluate(in Input) Result { + byService := map[string][]docker.Container{} + for _, c := range in.Containers { + s := c.Labels[stack.LabelService] + byService[s] = append(byService[s], c) + } + var res Result + var firstFailed, firstPending *Failure + allReady := true + for _, name := range in.Services { + svc, out, why := evaluateService(in, name, byService[name]) + res.Services = append(res.Services, svc) + switch out { + case failed: + if firstFailed == nil { + firstFailed = why + } + allReady = false + case pending: + if firstPending == nil { + firstPending = why + } + allReady = false + } + } + switch { + case firstFailed != nil: + res.Verdict, res.Failure = Failed, firstFailed + case allReady: + res.Verdict = Ready + case in.Final: + res.Verdict, res.Failure = TimedOut, firstPending + default: + res.Verdict = Starting + } + return res +} + +// evaluateService looks at every container of a service; the worst one decides. +func evaluateService(in Input, name string, containers []docker.Container) (Service, outcome, *Failure) { + svc := Service{Name: name} + route, routed := in.Routes[name] + if routed { + answering := in.Answering[name] + svc.Answering = &answering + } + if len(containers) == 0 { + svc.State = "missing" + return svc, failed, &Failure{Service: name, Reason: Missing, Hint: "the service has no container; run siding up"} + } + worst, why := ready, (*Failure)(nil) + for _, c := range containers { + svc.State, svc.Health = c.State, c.Health + out, f := evaluateContainer(in, name, c, route, routed) + if out > worst { + worst, why = out, f + } + if worst == failed { + break + } + } + svc.Ready = worst == ready + return svc, worst, why +} + +func evaluateContainer(in Input, name string, c docker.Container, route Route, routed bool) (outcome, *Failure) { + failure := func(reason, format string, args ...any) (outcome, *Failure) { + return failed, &Failure{Service: name, Reason: reason, Hint: fmt.Sprintf(format, args...)} + } + notReady := func() (outcome, *Failure) { + health := "" + if c.Health != "" { + health = ", health " + c.Health + } + return pending, &Failure{Service: name, Reason: NotReady, + Hint: fmt.Sprintf("not ready after %s (state %s%s); raise --timeout, or see its last log lines", in.Timeout, c.State, health)} + } + restarts := c.RestartCount - in.RestartBase[c.ID] + switch { + case c.State == "exited" || c.State == "dead": + if c.ExitCode == 0 && c.State == "exited" { + return ready, nil // a one-off service such as a migration has done its job + } + if c.OOMKilled { + return failure(Exited, "exited with code %d, killed for lack of memory; give it more memory or see its last log lines", c.ExitCode) + } + return failure(Exited, "exited with code %d; see its last log lines", c.ExitCode) + case restarts >= restartLimit || (c.State == "restarting" && in.Final): + return failure(Restarting, "keeps restarting (%d restarts, exit code %d); see its last log lines", c.RestartCount, c.ExitCode) + case c.Health == "unhealthy": + if c.HealthOutput != "" { + return failure(Unhealthy, "its healthcheck fails: %s", c.HealthOutput) + } + return failure(Unhealthy, "its healthcheck fails; see its last log lines") + case c.State != "running" || c.Health == "starting": + return notReady() + case routed && !in.Answering[name]: + return pending, &Failure{Service: name, Reason: NotAnswering, + Hint: fmt.Sprintf("it runs, but %s does not answer through the proxy; check that it listens on port %d on 0.0.0.0, not only on 127.0.0.1", route.URL, route.Port)} + } + return ready, nil +} diff --git a/internal/verdict/verdict_test.go b/internal/verdict/verdict_test.go new file mode 100644 index 0000000..8de31e3 --- /dev/null +++ b/internal/verdict/verdict_test.go @@ -0,0 +1,144 @@ +package verdict + +import ( + "testing" + "time" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/stack" +) + +func ct(service, state, health string, exit int) docker.Container { + return docker.Container{ID: service, Labels: map[string]string{stack.LabelService: service}, State: state, Health: health, ExitCode: exit} +} + +func TestEvaluate(t *testing.T) { + t.Parallel() + routes := map[string]Route{"web": {URL: "http://shop.main.localhost/", Port: 3000}} + tests := []struct { + name string + services []string + containers []docker.Container + answering map[string]bool + base map[string]int + final bool + verdict string + service string + reason string + }{ + { + name: "healthy, answering and done", + services: []string{"db", "migrate", "web"}, + containers: []docker.Container{ct("db", "running", "healthy", 0), ct("migrate", "exited", "", 0), ct("web", "running", "", 0)}, + answering: map[string]bool{"web": true}, + verdict: Ready, + }, + { + name: "running but not answering", + services: []string{"web"}, + containers: []docker.Container{ct("web", "running", "", 0)}, + verdict: Starting, + }, + { + name: "healthcheck starting", + services: []string{"db"}, + containers: []docker.Container{ct("db", "running", "starting", 0)}, + verdict: Starting, + }, + { + name: "exited non-zero", + services: []string{"api", "db"}, + containers: []docker.Container{ct("api", "exited", "", 1), ct("db", "running", "starting", 0)}, + verdict: Failed, service: "api", reason: Exited, + }, + { + name: "unhealthy", + services: []string{"db"}, + containers: []docker.Container{{ID: "db", Labels: map[string]string{stack.LabelService: "db"}, State: "running", Health: "unhealthy", HealthOutput: "pg_isready: no response"}}, + verdict: Failed, service: "db", reason: Unhealthy, + }, + { + name: "missing", + services: []string{"db", "web"}, + containers: []docker.Container{ct("web", "running", "", 0)}, + answering: map[string]bool{"web": true}, + verdict: Failed, service: "db", reason: Missing, + }, + { + name: "restart loop", + services: []string{"api"}, + containers: []docker.Container{{ID: "api", Labels: map[string]string{stack.LabelService: "api"}, State: "running", RestartCount: 5}}, + base: map[string]int{"api": 2}, + verdict: Failed, service: "api", reason: Restarting, + }, + { + name: "restarts from before the wait do not count", + services: []string{"api"}, + containers: []docker.Container{{ID: "api", Labels: map[string]string{stack.LabelService: "api"}, State: "running", RestartCount: 5}}, + base: map[string]int{"api": 5}, + verdict: Ready, + }, + { + name: "timed out not answering", + services: []string{"web"}, + containers: []docker.Container{ct("web", "running", "", 0)}, + final: true, + verdict: TimedOut, service: "web", reason: NotAnswering, + }, + { + name: "timed out starting", + services: []string{"db", "web"}, + containers: []docker.Container{ct("db", "running", "starting", 0), ct("web", "running", "", 0)}, + final: true, + verdict: TimedOut, service: "db", reason: NotReady, + }, + { + name: "the first failure in compose order is reported", + services: []string{"a", "b"}, + containers: []docker.Container{ct("b", "exited", "", 2), ct("a", "exited", "", 3)}, + verdict: Failed, service: "a", reason: Exited, + }, + { + name: "a scaled service: one bad replica fails it", + services: []string{"worker"}, + containers: []docker.Container{ + {ID: "w1", Labels: map[string]string{stack.LabelService: "worker"}, State: "running"}, + {ID: "w2", Labels: map[string]string{stack.LabelService: "worker"}, State: "exited", ExitCode: 137}, + }, + verdict: Failed, service: "worker", reason: Exited, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + res := Evaluate(Input{Services: tt.services, Containers: tt.containers, Routes: routes, + Answering: tt.answering, RestartBase: tt.base, Final: tt.final, Timeout: time.Minute}) + if res.Verdict != tt.verdict { + t.Fatalf("verdict %s, want %s (failure %+v)", res.Verdict, tt.verdict, res.Failure) + } + if tt.reason == "" { + if res.Failure != nil { + t.Errorf("unexpected failure %+v", res.Failure) + } + return + } + if res.Failure == nil || res.Failure.Service != tt.service || res.Failure.Reason != tt.reason || res.Failure.Hint == "" { + t.Errorf("failure %+v, want service %s, reason %s and a hint", res.Failure, tt.service, tt.reason) + } + }) + } +} + +func TestHints(t *testing.T) { + t.Parallel() + res := Evaluate(Input{Services: []string{"db"}, Containers: []docker.Container{ + {ID: "db", Labels: map[string]string{stack.LabelService: "db"}, State: "running", Health: "unhealthy", HealthOutput: "pg_isready: no response"}, + }}) + if want := "its healthcheck fails: pg_isready: no response"; res.Failure.Hint != want { + t.Errorf("hint %q, want %q", res.Failure.Hint, want) + } + res = Evaluate(Input{Services: []string{"api"}, Containers: []docker.Container{ct("api", "exited", "", 1)}}) + if want := "exited with code 1; see its last log lines"; res.Failure.Hint != want { + t.Errorf("hint %q, want %q", res.Failure.Hint, want) + } +} diff --git a/justfile b/justfile index f16cd76..46cdd02 100644 --- a/justfile +++ b/justfile @@ -25,8 +25,9 @@ test: openspec validate --all --strict --no-interactive # Rewrite golden files from the current code after an intended change +# (every package with golden files; a package without the -update flag would fail on it) golden: - go test ./internal/cli -update + go test ./internal/cli ./internal/stack -update # Build a static siding binary for this container (linux) into bin/ build: @@ -35,3 +36,7 @@ build: # Inner daemon: a compose project with a healthcheck, ports reset by an override, Traefik routing smoke-docker: hack/smoke/docker.sh + +# Inner daemon: siding's own commands end to end (setup, up --wait, routing, verdicts, ps, down) +smoke-stacks: + hack/smoke/stacks.sh diff --git a/openspec/changes/phase-1-stacks-and-routing/.openspec.yaml b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/.openspec.yaml similarity index 70% rename from openspec/changes/phase-1-stacks-and-routing/.openspec.yaml rename to openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/.openspec.yaml index 0d31f8b..1bb6456 100644 --- a/openspec/changes/phase-1-stacks-and-routing/.openspec.yaml +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/.openspec.yaml @@ -1,3 +1,2 @@ schema: spec-driven created: 2026-10-28 -skip_specs: true diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/design.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/design.md new file mode 100644 index 0000000..53dcd8a --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/design.md @@ -0,0 +1,223 @@ +# Design + +## Context + +Phase 0 left a Cobra root with `version`, the document types in `internal/cli/json.go`, an +`Error` type with a stable code, golden tests, and a shell smoke test showing that the building +blocks work: `ports: !reset []`, Traefik with the Docker provider on the network `siding`, Host +and PathPrefix rules (`hack/smoke/testdata/override.yml` is the hand-written model of what +`siding up` must generate). Motivation and scope: see proposal.md; behaviour: see the delta +specs. Constraints from CLAUDE.md: Docker is the database, the compose file is never edited, no +published ports, runtime truth, the same output for agent and human, no package-level test +seams. + +## Goals / Non-Goals + +**Goals:** +- Every package that touches the machine (git, `docker compose`, Docker API, HTTP probe, clock, + browser) sits behind a small interface in `cli.Options`, so all commands, verdicts and + overrides are tested without Docker, in parallel, against golden files. +- One end-to-end test of the real thing against the inner daemon (`just smoke-stacks`). + +**Non-Goals:** +- Acting on a stack other than the current worktree's (`down `, `gc`): v0.2. +- Memory budget, exit 3, web overview: v0.2. Snapshots, seeds: v0.3. Idle stop: v0.4. +- Reaching a published database port from the host: ports are reset; `docker compose exec` + or a routed admin UI is the way in v0.1. +- Path stripping, TLS, WebSocket specifics: Traefik passes WebSockets through as it is. + +## Decisions + +### D1. Package layout +`internal/git` (worktree, branch, common dir, dirty, merged), `internal/compose` (run +`docker compose`, parse `config --format json`), `internal/docker` (Engine API: containers, +networks, volumes, logs, stats), `internal/stack` (identity, slugs, labels, override +generation, routing table: pure functions), `internal/verdict` (from service states and probe +results to a verdict: pure), `internal/proxy` (the embedded proxy compose file, setup), and +`internal/cli` for commands and documents. Pure packages get table-driven tests; the others are +thin and covered by the end-to-end test. + +### D2. Reading the user's compose file through `docker compose config` +siding runs `docker compose -f config --format json` and reads services, ports, +`expose`, labels, networks, `network_mode`, healthchecks, volumes and `env_file` from the +normalised result. Compose does interpolation, profiles, `include` and `extends`; siding parses +none of it itself. Alternative: the compose-go library in process. Lost: it would pin a second +Compose implementation into the binary that can disagree with the user's `docker compose` +(principle 8, standard building blocks). + +### D3. Docker Engine API through `github.com/moby/moby/client` +State (containers by label, health, exit codes, published ports, logs, one-shot stats) is read +through the Engine API. `github.com/docker/docker` is frozen at v28; the client now lives in +its own module. Lifecycle (`up`, `down`, the proxy) goes through `docker compose`, as the +conventions say. Alternative: parsing `docker inspect` output. Lost: slower, and more text +formats to keep stable. + +### D4. Proxy as Compose project `siding` (maintainer's decision: port 80, `--port`) +`siding setup` writes the embedded compose file to `~/.siding/proxy/compose.yml` (a losable +cache, rewritten on every setup) and runs it as project `siding`, container `siding-proxy`, +image pinned in the binary (`traefik:v3.7.16`, the same as the devcontainer's +`SIDING_TRAEFIK_IMAGE`), published on `127.0.0.1:`, port 80 by default. Traefik flags: +Docker provider, `exposedbydefault=false`, `network=siding`, and a constraint +`LabelRegex(\`dev.siding.repo\`, \`.+\`)`, so foreign Traefik labels are ignored. The port is +read back from the container's port binding (Docker is the database); no file holds it. +In the devcontainer the developer runs `siding setup --port 8080`, the forwarded port. + +Alternatives for the port, asked: 8080 by default (every URL carries `:8080`); 80 with an +automatic fallback to 8080 (convenient, less predictable). Alternative for the mechanism: +a plain container created through the API. Lost: the proxy then is not a Compose project, +and "removable, plain Compose" (principle 9) would not hold for it. + +### D5. Identity (maintainer's decision: repository name from the main worktree) +Repository name = basename of the parent of `git rev-parse --path-format=absolute +--git-common-dir`, overridable with `--repo`. Slugs as in the spec `stack-identity`; 40 +characters keep `...localhost` labels well under DNS's 63 and project +names short. Alternative, asked: the name of the `origin` remote (stable when the folder is +renamed, but needs a remote and differs from what the user sees in the file system). + +A detached HEAD is an error (`NoBranch`), because a slug from a commit would make a new stack +for every commit. Collisions (`feature/a`, `feature-a`) are detected through the labels of the +existing project, not prevented by hashing every slug, so normal names stay readable. + +### D6. Machine id +`sha256("siding-machine-id:" + os id)`, first 16 hex digits; the os id is `/etc/machine-id` +on Linux and `IOPlatformUUID` from `ioreg` on macOS. Without one (many containers), a random id +in `~/.siding/machine-id`. Hashing keeps the raw id, which other software treats as +confidential, out of labels. Losing the file changes the id; in v0.1 that only changes a label +nobody filters on yet, and v0.5 can make the fallback durable if remote hosts need it. +Alternative: always a generated file. Lost: reinstalling or clearing `~/.siding` would change +the id even where the OS has a stable one. + +### D7. The override (maintainer's decision: `dev.siding.port` for ambiguous ports) +Generated from the normalised config, written as YAML with `!reset` tags: +- every service: `ports: !reset []`, labels `dev.siding.*`; +- every HTTP service not hidden: `networks:` = its normalised networks plus `siding` (a list + in an override replaces the default network otherwise), `traefik.enable=true`, + `traefik.docker.network=siding`, one router `-` for + `Host(\`...localhost\`)`, and for a route path a second router + `--route` for `Host(\`..localhost\`)` plus + `PathPrefix(\`\`)` (none for `/`); Traefik's default priority (rule length) makes the + longest prefix win; +- a headers middleware on every siding router that adds `X-Siding-Service: /` + to responses (D9); +- top-level: labels on the project's own networks (`default` included) and named volumes, not + on external ones; the network `siding` as external. + +Network and volume labels never change between two `up`s of one stack: Compose 5.6 recreates +a network whose labels changed, stopping its containers, and keeps a volume's first labels. +So `dev.siding.agent`, which may differ from one `up` to the next, goes on containers only; +the other labels are fixed for a project (D5 refuses a second branch or worktree). Compose +always runs with stdin closed, so a confirmation prompt fails instead of hanging. +Golden files under `internal/stack/testdata/override/` hold input config and expected override. + +HTTP detection, asked: exactly one container port from `ports` targets and `expose`, or +`dev.siding.port`. Alternatives: the first port wins (a database with a published port would +be routed by accident); only services with a route label (drops the service host of the +roadmap). The new label changes the convention list in CLAUDE.md and `openspec/config.yaml`. + +A path prefix is passed on unchanged (no StripPrefix): the common dev setups (Vite, nginx, +Next.js rewrites) mount an API under `/api` and expect `/api` to arrive. A stripping option +can be added later as a label value without breaking this one. + +### D8. `.siding/.gitignore` +siding writes `.siding/.gitignore` with `*`, which ignores the directory including itself, so +`.siding/` never makes the worktree dirty and the user's `.gitignore` is never touched. +Alternative: ask users to add `.siding/` to `.gitignore`. Lost: one more setup step, and a +forgotten one would make `down --volumes` refuse with `DirtyWorktree`. + +### D9. Readiness probe through the proxy +siding sends `GET ` with `Host: ` to `127.0.0.1:` (no DNS +needed) for every service with a route path, and for `/` routes to `/`. A response counts as +the service's own when it carries `X-Siding-Service` for that service and its status is not +502, 503 or 504. Traefik's own 404 for a missing router has no such header; its 502/503/504 +for an unreachable backend do not count. The probe follows no redirects; a 3xx counts. +Alternatives: any status below 500 (cannot tell Traefik's 404 from the app's); asking +Traefik's API whether the router exists (needs a second published port). The end-to-end test +shows that Traefik 3.7 adds the header to its own 502 as well (the middleware runs before the +backend fails), so the status rule is what rules those out; its 404 for an unknown host has +no header. + +### D10. Verdict +A pure function from (services in the config, containers with state, health, exit code, +restart count, probe results, elapsed time, timeout) to the verdict. Order of evaluation: +`Missing` (no container after `up` has finished), `Exited` (non-zero), `Unhealthy`, +`Restarting` (restart count grew by 3 or more while waiting, or state `restarting` at the end), +then `Starting` until healthy, running or exited 0 and answering. `--wait` polls every second. +A service that exits 0 counts as done (migrations, seeds). The first failing service in +compose order is reported; others appear in `services` with their state. The hint is a fixed +text per reason, completed with the exit code or the last healthcheck output. Log tail: 20 +lines from the Docker API, redacted (D12). + +`status` without `--wait` and with `Starting` exits 2: "not ready now" is the same answer as a +wait of zero. Alternative: exit 0 with the verdict in the document. Lost: a script checking +only the exit code would take Starting for Ready. + +### D11. `down` (maintainer's decision: containers go, volumes stay) +`docker compose -p down --remove-orphans` (by name, works without the compose file); +`--volumes` adds `-v` after the guard: `git status --porcelain` empty, and the branch tip an +ancestor of the default branch (`refs/remotes/origin/HEAD`, else local `main`, else +`master`, else unmerged). `--dry-run` lists containers, networks and, with `--volumes`, +volumes by label through the API. Alternatives, asked: `down` always removes volumes and is +always guarded (an agent could not stop its stack in a dirty worktree without `--force`); +`down` only stops (removing would wait for `gc`). This makes the guardrail "never lose work" +precise: it guards a stack's data and worktrees, not containers. CLAUDE.md and +`openspec/config.yaml` change with it. A squash-merged branch counts as unmerged; `--force` +is the way until `gc` (v0.2) learns more. + +### D12. Redaction +Values from the worktree's `.env` and every service's `environment` in the normalised config, +4 characters or longer, longest first, are replaced by `***` in every log line siding puts +into a document. The normalised config no longer names `env_file`s (Compose 5.6 inlines their +values into `environment`, also with `--no-normalize`), so redacting all of `environment` +covers them and inline values too. Numbers and `true`, `false`, `yes`, `no`, `on`, `off` are +left, as are values shorter than 4 characters, since `***` for them would wreck every line. +The `.env` is parsed with Compose's rules for `KEY=VALUE`, `export`, quotes and comments; no +values are logged or cached. Text output of `siding logs` stays raw: it is the user's own terminal, as with +`docker compose logs`. + +### D13. `siding skill` (decided here, open for review) +The skill text is embedded in the binary (`internal/cli/skill/SKILL.md`, golden-tested) and +printed; siding writes no file into the user's repository. Alternatives: `siding skill +install` writing `.claude/skills/siding/SKILL.md` (one more file siding writes into a user's +repository); a Claude Code plugin marketplace entry (an extra channel to keep in step with the +binary). Printing keeps the text and the binary in step and lets the user choose where it goes. + +### D14. Test seams +`cli.Options` gains `Git`, `Compose`, `Docker`, `Probe`, `Browser`, `Clock`, `Home`, `Env` +and `Proxy` (container name, network, project, image, default port). `main` fills them with +real implementations; unit tests use fakes. The end-to-end tests under `internal/e2e` with +`//go:build docker` run commands in process with `Proxy` set to `siding-test-proxy` on the +network `siding-test` at port 18080, in temporary git repositories whose `--repo` starts with +`siding-test-`, and remove everything by exact name in `t.Cleanup`. So the binary has no hidden +test switches, and a developer's `siding-proxy` in the devcontainer is never touched. + +### D15. No silent port remap in the devcontainer (maintainer's decision) +VS Code forwards a port to another host port when the requested one is taken, without asking: +in the manual test it forwarded 8080 to 8081, and every address siding printed (`:8080`) was +wrong. `"requireLocalPort": true` on 8080 makes VS Code report the conflict instead. Someone +who cannot free 8080 changes the port in `forwardPorts` and `portsAttributes` and runs +`siding setup --port` with the same number. Alternatives: leave the remap and document "look +in the PORTS tab" (every printed address stays wrong); let siding learn the host port (siding +cannot see VS Code's forwarding, and v0.1 has no configuration for an address prefix). + +## Risks / Trade-offs + +- [Some tools do not resolve `*.localhost`] → browsers and curl do; siding's own probe uses + `127.0.0.1` with a Host header; README notes `curl` and `/etc/hosts` for others. +- [Port 80 is taken or not bindable] → `PortInUse` with the hint `--port 8080`; addresses then + carry the port. +- [Users lose host access to published database ports] → documented in README with + `docker compose exec`; a routed TCP option is out of scope for v0.1. +- [Traefik adds `X-Siding-Service` to its own errors] → the status check excludes 502/503/504; + the end-to-end test pins the behaviour. +- [`docker compose config` differs between Compose versions] → siding requires 2.24 and checks + the version once per command that runs Compose; golden tests cover the parsed fields. +- [Squash merges count as unmerged] → `--force`, and the hint says so. +- [Redaction misses secrets that are not in `.env` files, e.g. inline `environment:` values] + → the guarantee is stated as `.env` and `env_file` values only, which is what the safety + rule asks; inline values are visible in the compose file anyway. + +## Migration Plan + +Nothing to migrate: v0.1 is the first release with stacks. Uninstalling leaves plain Compose +projects (`-` and `siding`) that keep running and are removed with +`docker compose -p down`. diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/proposal.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/proposal.md new file mode 100644 index 0000000..d9b825e --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/proposal.md @@ -0,0 +1,67 @@ +# Proposal + +## Why + +siding exists to answer "does what my agent built actually run?". Today it only prints its +version. v0.1 is the smallest siding that answers the question: every worktree runs its own +stack from the unchanged compose file, every stack is reachable by name through one proxy, and +an agent can wait for a verdict, Ready or Failed with the reason and the logs, as versioned +JSON. + +## What Changes + +- `siding setup [--port ]`: starts the shared proxy `siding-proxy` (Traefik, Docker + provider) on the network `siding`, listening on `127.0.0.1:80` by default. Idempotent. +- `siding up [--wait] [-- ]`: in the current worktree, writes + `.siding/override.yml` (ports reset, network `siding`, Traefik and `dev.siding.*` labels) and + runs the user's compose file as project `-`. With `--wait` it ends with the + verdict of `status --wait`. +- `siding status [--wait [--timeout ]]`: the verdict Ready, Starting, Failed or TimedOut; + a failure names the service, the reason, a hint and the last log lines of that service. +- `siding ps`: every stack on the machine, read from labels and the Docker API, with its + state, address and memory use. +- `siding logs [service…]`, `siding open [service]`: the stack's logs and its address. +- `siding down [--volumes] [--force] [--dry-run]`: removes the stack's containers and network. + Volumes go only with `--volumes`, which refuses when the branch is unmerged or the worktree + dirty, unless `--force`. +- Routing: `..localhost` for the stack; `dev.siding.route` (`/`, + `/`, `false`) mounts services on it; every HTTP service also answers at + `...localhost`. A new service label `dev.siding.port` picks the + port of a service with several. +- `siding skill`: prints a Claude Code skill that tells an agent how to start its stack and + wait for the verdict. +- Every command has `--json`, with the new kinds `Setup`, `Stack`, `Status`, `StackList`, + `Logs`, `Address`, `Down` and `Skill`, and new stable error codes. +- The guardrail "never lose work" is made precise: it guards removing a stack's data (its + volumes) or a worktree; removing containers loses nothing and is not guarded. CLAUDE.md and + the project context change with it, as does the convention list (`dev.siding.port`). + +## Capabilities + +### New Capabilities +- `cli-contract`: the `--json` documents, the `Error` document, exit codes and `siding version` + (exists since Phase 0, now specified). +- `stack-identity`: how a worktree maps to repo name, branch slug, Compose project, host names, + machine id and the `dev.siding.*` labels. +- `proxy`: `siding setup` and the shared Traefik `siding-proxy` on the network `siding`. +- `routing`: which services are HTTP services, how route labels map to hosts and paths, and + `siding open`. +- `stack-lifecycle`: `siding up`, `siding down`, `siding logs`, the override file and the + guard that keeps unmerged or dirty work. +- `stack-status`: the verdict of `siding status`, its reasons, hints and log tail. +- `stack-list`: `siding ps`. +- `agent-skill`: the Claude Code skill printed by `siding skill`. + +### Modified Capabilities +- `dev-environment`: a second smoke test, `just smoke-stacks`, runs siding's own commands + end to end against the inner daemon, in CI on pull requests and on `main`. + +## Impact + +- Go code under `internal/` (git, compose, Docker, routing, verdict) and new commands in + `internal/cli`; the Docker Engine Go SDK as a new dependency. +- `docker compose` (>= 2.24) and `git` become runtime requirements of siding. +- New golden files per document kind and for generated overrides; an end-to-end test under a + `docker` build tag; `justfile` and CI gain `smoke-stacks`. +- CLAUDE.md, README and `openspec/config.yaml`: status, the precise guardrail, the label + `dev.siding.port`, the proxy port. diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/agent-skill/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/agent-skill/spec.md new file mode 100644 index 0000000..56d4981 --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/agent-skill/spec.md @@ -0,0 +1,26 @@ +# Spec Delta + +## Purpose + +Defines the Claude Code skill siding ships, so that a coding agent starts its worktree's stack +and waits for the verdict instead of guessing whether its work runs. + +## ADDED Requirements + +### Requirement: siding skill prints the skill +`siding skill` SHALL print a Claude Code skill (`SKILL.md` with name, description and body) on +stdout and write no file. With `--json` it SHALL print a `Skill` document holding the same text. + +#### Scenario: Install the skill +- **WHEN** a user runs `mkdir -p .claude/skills/siding && siding skill > .claude/skills/siding/SKILL.md` +- **THEN** the file is a valid skill named `siding`, and siding wrote nothing itself + +### Requirement: What the skill teaches +The skill SHALL tell the agent to run `siding up --wait --json` after a change that affects the +running app, to read the verdict and exit code, to fix the named service from the failure's +reason, hint and log tail, and to report the address when Ready. It MUST NOT tell the agent to +use `--force`, `--volumes` or `siding share`. + +#### Scenario: The commands in the skill +- **WHEN** the printed skill is read +- **THEN** it names `siding up --wait --json`, `siding status --json`, `siding logs --json` and the exit codes 0, 1 and 2, and none of `--force`, `--volumes` or `share` diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/cli-contract/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/cli-contract/spec.md new file mode 100644 index 0000000..7fc3f5c --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/cli-contract/spec.md @@ -0,0 +1,76 @@ +# Spec Delta + +## Purpose + +Defines what agents and scripts can rely on when they call siding: one versioned JSON document +per command with `--json`, stable error codes and fixed exit codes. + +## ADDED Requirements + +### Requirement: One JSON document per command +Every siding command SHALL accept `--json` and then print exactly one JSON document on stdout +and nothing else there. The document SHALL carry `apiVersion: siding.dev/v1alpha1` and a +`kind`. Empty fields MUST be left out, never printed as `null`. + +#### Scenario: A command with --json +- **WHEN** `siding version --json` runs +- **THEN** stdout holds one document with `apiVersion` `siding.dev/v1alpha1` and `kind` `Version`, and nothing else + +#### Scenario: Progress of docker compose +- **WHEN** a command with `--json` runs `docker compose`, which reports progress +- **THEN** that progress goes to stderr, and stdout still holds only the document + +#### Scenario: An empty field +- **WHEN** a document has a field without a value, such as a commit unknown to the build +- **THEN** the field is missing from the document + +### Requirement: Compatible within a version +Within `siding.dev/v1alpha1` a document field and an error code MUST NOT be removed, renamed +or given another meaning; new fields, kinds and codes MAY be added. + +#### Scenario: A field is added +- **WHEN** a later release adds a field to the `Status` document +- **THEN** documents still carry `apiVersion` `siding.dev/v1alpha1` and every earlier field with its meaning + +### Requirement: Error document +A command that fails SHALL exit 1. With `--json` it SHALL print a document of kind `Error` +with a stable `code` and a human-readable `message`; without `--json` it SHALL print +`error: ` on stderr, followed by a hint line when there is one. + +#### Scenario: An unknown flag with --json +- **WHEN** `siding version --json --nope` runs +- **THEN** it exits 1 and prints an `Error` document with `code` `InvalidArguments`, even though flag parsing failed before `--json` was read + +#### Scenario: A failure without --json +- **WHEN** `siding up` runs outside a git repository +- **THEN** it exits 1 and prints `error: …` and a hint on stderr, and nothing on stdout + +#### Scenario: The error codes +- **WHEN** a command fails +- **THEN** its code is one of `InvalidArguments`, `NotAGitRepository`, `NoBranch`, `ComposeFileNotFound`, `ComposeFailed`, `DockerUnavailable`, `ProxyNotRunning`, `PortInUse`, `InvalidRoute`, `AmbiguousPort`, `SlugCollision`, `StackNotFound`, `UnmergedBranch`, `DirtyWorktree`, or `Internal` for an unexpected error + +### Requirement: Exit codes +siding SHALL exit 0 on success and when a stack is Ready, 1 when a command fails or a stack is +Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`), 3 for +NoCapacity, which v0.1 never returns, and 130 without a message when interrupted. + +#### Scenario: A verdict decides the exit code +- **WHEN** `siding status --wait` ends with the verdict Failed +- **THEN** it exits 1, and with Ready it exits 0 + +#### Scenario: Interrupted +- **WHEN** the user presses Ctrl-C during `siding logs -f` +- **THEN** siding stops docker compose and exits 130 without an error message + +### Requirement: Version +`siding version` SHALL print the version, the commit (when the build knows it), the Go version +and the platform; with `--json` as a document of kind `Version` with the fields `version`, +`commit`, `goVersion` and `platform`. + +#### Scenario: Text output +- **WHEN** `siding version` runs in a build of commit `0123456789abcdef…` +- **THEN** it prints one line such as `siding v0.1.0 (0123456789ab) go1.27.3 linux/arm64` + +#### Scenario: An argument +- **WHEN** `siding version extra` runs +- **THEN** it fails with `InvalidArguments` diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/dev-environment/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/dev-environment/spec.md new file mode 100644 index 0000000..d4f6373 --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/dev-environment/spec.md @@ -0,0 +1,41 @@ +# Spec Delta + +## ADDED Requirements + +### Requirement: Stack smoke test +`just smoke-stacks` SHALL run siding's own commands end to end against the inner daemon: +`setup`, `up --wait`, routing by host and path, a `Failed` verdict with log tail, `ps`, `down` +and the refusal of `down --volumes`. Its proxy, network and projects MUST carry the prefix +`siding-test-`, and its proxy MUST listen on 18080. + +#### Scenario: Stack smoke test in CI +- **WHEN** a pull request is opened or pushed to, or a commit is pushed to `main` +- **THEN** CI runs `just smoke-stacks` inside the devcontainer, against its own daemon + +#### Scenario: It leaves nothing behind +- **WHEN** `just smoke-stacks` ends, passing or failing +- **THEN** no container, network or volume whose name starts with `siding-test-` is left in the inner daemon, and `siding-proxy` and the network `siding` are untouched + +#### Scenario: Not part of level 1 +- **WHEN** `just test` runs without a Docker daemon +- **THEN** it passes, because the end-to-end tests only build with the `docker` build tag + +## MODIFIED Requirements + +### Requirement: Devcontainer settings +The devcontainer SHALL be named `siding-devcontainer`, set `SIDING_DEVCONTAINER=1`, forward port +8080 inside to 8080 on the host for the proxy and never to another host port, and set +`git config worktree.useRelativePaths true` when it is created, so worktrees work both inside +the container and on the host. + +#### Scenario: A worktree made in the container +- **WHEN** `git worktree add` runs inside the devcontainer +- **THEN** the worktree's `.git` file and the repository's back-reference use relative paths + +#### Scenario: The host's Git is older than 2.48 +- **WHEN** a worktree with relative paths exists, and Git before 2.48 on the host opens the repository +- **THEN** it refuses with "unknown repository extension found: relativeworktrees", so the host needs Git 2.48 or later + +#### Scenario: Port 8080 is taken on the host +- **WHEN** the devcontainer opens while another process holds port 8080 on the host +- **THEN** VS Code reports that it cannot forward 8080, instead of forwarding to another port that the addresses siding prints do not name diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/proxy/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/proxy/spec.md new file mode 100644 index 0000000..f95cad9 --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/proxy/spec.md @@ -0,0 +1,67 @@ +# Spec Delta + +## Purpose + +Defines the one shared proxy through which every stack is reached: `siding-proxy`, a Traefik +on the network `siding`, started by `siding setup` and bound to the loopback interface only. + +## ADDED Requirements + +### Requirement: Setup starts the proxy +`siding setup` SHALL create the network `siding` if it is missing and run the Compose project +`siding` with one container `siding-proxy`: Traefik with the Docker provider on the network +`siding`, restarting unless stopped. It SHALL print the proxy's address, with `--json` as a +`Setup` document. + +#### Scenario: First setup +- **WHEN** `siding setup` runs on a machine without the proxy +- **THEN** the network `siding` and the running container `siding-proxy` exist, and it prints `http://*.localhost` as the address + +#### Scenario: JSON +- **WHEN** `siding setup --json` runs +- **THEN** it prints a `Setup` document with the proxy's container name, network, image and port + +### Requirement: Loopback port +The proxy SHALL publish exactly one port, on `127.0.0.1` only, `80` unless `--port ` names +another. siding MUST read the proxy's port from the running container, not from a file. + +#### Scenario: Default port +- **WHEN** `siding setup` has run +- **THEN** `siding-proxy` publishes `127.0.0.1:80` and no other port, and nothing listens for it on other interfaces + +#### Scenario: Another port +- **WHEN** `siding setup --port 8080` has run and `siding open` runs later +- **THEN** the address it opens is `http://:8080` + +#### Scenario: The port is taken +- **WHEN** another process holds `127.0.0.1:80` and `siding setup` runs +- **THEN** it fails with `PortInUse` and a hint to use `--port` + +### Requirement: Setup is idempotent +`siding setup` SHALL change nothing when the proxy already runs as requested, and SHALL +recreate only the proxy container when its port or image differs. Running stacks MUST keep +running and be routed again once the proxy is back. + +#### Scenario: Second run +- **WHEN** `siding setup` runs twice +- **THEN** the second run reports the proxy as already running and recreates nothing + +#### Scenario: Port change with running stacks +- **WHEN** a stack runs and `siding setup --port 8080` follows `siding setup` +- **THEN** the stack's containers are not restarted and it answers on port 8080 + +### Requirement: Only siding stacks are routed +The proxy MUST route only containers that carry `dev.siding.repo` and enable Traefik, so that +other containers with Traefik labels on the same daemon are never exposed through it. + +#### Scenario: A foreign Traefik label +- **WHEN** a container without siding labels carries `traefik.enable=true` and a Host rule +- **THEN** the proxy answers 404 for that host + +### Requirement: Stacks need the proxy +`up` and `status` SHALL fail with `ProxyNotRunning` and the hint `siding setup` when +`siding-proxy` is not running. + +#### Scenario: up before setup +- **WHEN** `siding up` runs on a machine without the proxy +- **THEN** it fails with `ProxyNotRunning` before starting any container diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/routing/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/routing/spec.md new file mode 100644 index 0000000..645e64c --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/routing/spec.md @@ -0,0 +1,89 @@ +# Spec Delta + +## Purpose + +Defines how the services of a stack are reached through the proxy: which services count as +HTTP services, how the label `dev.siding.route` maps them to the stack's host and paths, and +how `siding open` finds the address. + +## ADDED Requirements + +### Requirement: HTTP services +A service SHALL be an HTTP service when the user's compose file gives it exactly one distinct +container port, from the targets of `ports` and from `expose`, or when its label +`dev.siding.port` names one. A service with several ports and no `dev.siding.port` is not one. + +#### Scenario: One published port +- **WHEN** a service publishes `"3000:3000"` +- **THEN** it is an HTTP service on container port 3000, and the port is no longer published + +#### Scenario: Several ports with the label +- **WHEN** a service exposes 3000 and 9229 and carries `dev.siding.port: "3000"` +- **THEN** it is an HTTP service on container port 3000 + +#### Scenario: No port +- **WHEN** a worker service has neither `ports` nor `expose` nor `dev.siding.port` +- **THEN** it is not routed + +### Requirement: Service host +Every HTTP service SHALL answer at `...localhost`, unless it +carries `dev.siding.route: "false"`. + +#### Scenario: An unlabelled HTTP service +- **WHEN** the service `mail` exposes 8025 and has no route label, in the stack `shop` on branch `main` +- **THEN** `mail.shop.main.localhost` reaches it on 8025 + +#### Scenario: Hidden +- **WHEN** a service carries `dev.siding.route: "false"` +- **THEN** no host or path reaches it through the proxy + +### Requirement: Route label +`dev.siding.route: "/"` SHALL make an HTTP service the root of the stack's host, and +`dev.siding.route: "/"` SHALL mount it on the stack's host under that path prefix, +passing the path on unchanged. The longest matching prefix MUST win. + +#### Scenario: Root and API +- **WHEN** `web` carries `dev.siding.route: "/"` and `api` carries `dev.siding.route: "/api"` in the stack `shop` on `main` +- **THEN** `shop.main.localhost/` reaches `web`, and `shop.main.localhost/api/users` reaches `api` with the path `/api/users` + +#### Scenario: No root +- **WHEN** no service carries `dev.siding.route: "/"` +- **THEN** a path on the stack's host that no route covers answers 404 + +### Requirement: Invalid routes +`siding up` SHALL fail with `InvalidRoute`, naming the service, and start nothing when a route +label is neither `false` nor an absolute path, when two services claim the same path, or when +a service with a route is not an HTTP service. + +#### Scenario: Two services on one path +- **WHEN** `web` and `admin` both carry `dev.siding.route: "/"` +- **THEN** `siding up` fails with `InvalidRoute` naming both + +#### Scenario: A route without a port +- **WHEN** a service carries `dev.siding.route: "/api"` but has no container port +- **THEN** `siding up` fails with `InvalidRoute` and a hint to add `expose` or `dev.siding.port` + +#### Scenario: A route with several ports +- **WHEN** a service carries `dev.siding.route: "/"` and exposes 3000 and 9229 without `dev.siding.port` +- **THEN** `siding up` fails with `AmbiguousPort` naming the ports + +### Requirement: Services that cannot join the proxy network +A service with a `network_mode` SHALL NOT be routed; `up` SHALL warn about it on stderr, and +fail with `InvalidRoute` when it carries a route path. + +#### Scenario: network_mode host +- **WHEN** a service uses `network_mode: host` and has no route label +- **THEN** `siding up` starts it unrouted and prints a warning + +### Requirement: Open +`siding open [service]` SHALL open the stack's address, or the service's host, in the default +browser; with `--print` or `--json` it SHALL only print it, as text or as an `Address` +document. The address MUST include the proxy's port unless it is 80. + +#### Scenario: Print the address +- **WHEN** the proxy listens on 8080 and `siding open --print` runs in the stack `shop` on `main` +- **THEN** it prints `http://shop.main.localhost:8080/` + +#### Scenario: Unknown service +- **WHEN** `siding open nope` runs and the stack has no HTTP service `nope` +- **THEN** it fails with `InvalidArguments` and lists the HTTP services diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-identity/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-identity/spec.md new file mode 100644 index 0000000..592d35d --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-identity/spec.md @@ -0,0 +1,101 @@ +# Spec Delta + +## Purpose + +Defines how siding derives a stack's identity from a git worktree: repository name, branch +slug, Compose project, host names, machine id and the labels that let Docker answer every +later question about the stack. + +## ADDED Requirements + +### Requirement: One stack per worktree +siding SHALL treat the git worktree containing the current directory as the stack's worktree, +identified by its absolute top-level path. Every command that acts on a stack MUST act on that +worktree's stack. + +#### Scenario: Run from a subdirectory +- **WHEN** `siding status` runs in `/src/shop-login/web/src`, inside the worktree `/src/shop-login` +- **THEN** it reports the stack of `/src/shop-login` + +#### Scenario: Outside a repository +- **WHEN** a stack command runs outside any git worktree +- **THEN** it fails with `NotAGitRepository` + +### Requirement: Repository name +The repository name SHALL be the name of the directory that holds the repository's common git +directory (the main worktree), the same for every worktree of the repository. `--repo ` +MUST override it. + +#### Scenario: Two worktrees of one repository +- **WHEN** `/src/shop` is the main worktree and `/src/shop-login` a linked worktree of it +- **THEN** both have the repository name `shop` + +#### Scenario: Override +- **WHEN** `siding up --repo store` runs in `/src/shop` +- **THEN** the repository name is `store` + +### Requirement: Slugs +The repository name and the branch SHALL each be turned into a slug: lowercase, every run of +characters other than `a-z` and `0-9` replaced by one `-`, no leading or trailing `-`. A slug +longer than 40 characters MUST be cut to 33 characters, followed by `-` and the first 6 hex +digits of the SHA-256 of the original name. + +#### Scenario: A branch with a slash +- **WHEN** the branch is `feature/Login_Form` +- **THEN** its slug is `feature-login-form` + +#### Scenario: A long branch +- **WHEN** the branch name is 60 characters long +- **THEN** its slug is 40 characters long and ends in `-` and 6 hex digits that differ for another branch with the same first 33 characters + +### Requirement: Branch required +The worktree SHALL be on a branch. In a detached HEAD a stack command MUST fail with `NoBranch` +and a hint to check out a branch. + +#### Scenario: Detached HEAD +- **WHEN** `siding up` runs in a worktree whose HEAD is detached +- **THEN** it fails with `NoBranch` and starts nothing + +### Requirement: Project name and host +The Compose project SHALL be named `-` and the stack's host SHALL be +`..localhost`. + +#### Scenario: Names +- **WHEN** the repository is `shop` and the branch `feature/login` +- **THEN** the project is `shop-feature-login` and the host `shop.feature-login.localhost` + +### Requirement: Slug collision +siding MUST NOT let two worktrees or branches share one project. If the project already exists +with another `dev.siding.worktree` or `dev.siding.branch` label, `up` MUST fail with +`SlugCollision`, naming the other worktree and branch, and change nothing. + +#### Scenario: feature/a and feature-a +- **WHEN** a stack for branch `feature/a` runs and `siding up` runs in a worktree on branch `feature-a` +- **THEN** it fails with `SlugCollision` and the running stack is untouched + +### Requirement: Labels +siding SHALL label every container, network and volume of a stack with `dev.siding.repo`, +`dev.siding.branch` (the branch name, not the slug), `dev.siding.worktree` (absolute path) and +`dev.siding.machine`, and its containers with `dev.siding.agent` when `--agent` or +`SIDING_AGENT` names one. External networks and volumes MUST NOT be relabelled. + +#### Scenario: Labels on a running stack +- **WHEN** `siding up` has run in `/src/shop-login` on branch `feature/login` +- **THEN** every container, the project's networks and its named volumes carry `dev.siding.repo=shop`, `dev.siding.branch=feature/login` and `dev.siding.worktree=/src/shop-login` + +#### Scenario: An agent name +- **WHEN** `SIDING_AGENT=claude siding up` runs +- **THEN** the containers also carry `dev.siding.agent=claude` + +### Requirement: Machine id +`dev.siding.machine` SHALL be a stable id of the machine siding runs on: derived from the +operating system's machine id by a one-way hash, never the raw id. Without an operating system +id, siding MUST generate one and keep it in `~/.siding/machine-id`. + +#### Scenario: Two runs +- **WHEN** `siding up` runs twice on the same machine +- **THEN** both stacks carry the same `dev.siding.machine` + +#### Scenario: No operating system id +- **WHEN** neither `/etc/machine-id` nor the macOS platform UUID is available +- **THEN** siding writes a random id to `~/.siding/machine-id` and uses it from then on diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md new file mode 100644 index 0000000..a742ffd --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-lifecycle/spec.md @@ -0,0 +1,125 @@ +# Spec Delta + +## Purpose + +Defines how a worktree's stack is started, inspected through its logs and taken down: the +user's compose file stays untouched, siding adds one override, and nothing removes unmerged or +uncommitted work without being forced. + +## ADDED Requirements + +### Requirement: Compose file +siding SHALL use the compose file Compose itself would pick in the worktree's top-level +directory (`compose.yaml`, `compose.yml`, `docker-compose.yaml`, `docker-compose.yml`), or the +one named by `--file`. Without one, stack commands MUST fail with `ComposeFileNotFound`. + +#### Scenario: The default file +- **WHEN** the worktree holds `compose.yaml` +- **THEN** `siding up` uses it + +#### Scenario: No compose file +- **WHEN** the worktree holds no compose file and `--file` is not given +- **THEN** `siding up` fails with `ComposeFileNotFound` + +### Requirement: The compose file is never edited +siding MUST NOT write to the user's compose file, its `.env` or any other file of the user. It +SHALL write only `.siding/override.yml` and `.siding/.gitignore` (containing `*`) in the +worktree, so that `.siding/` never makes the worktree dirty. + +#### Scenario: After up +- **WHEN** `siding up` has run in a clean worktree +- **THEN** `git status --porcelain` prints nothing and the compose file is byte for byte unchanged + +### Requirement: Override +The override SHALL reset every service's `ports` (`ports: !reset []`), add every routed +service to the network `siding` next to its own networks, and add the `dev.siding.*` and +Traefik labels. The user's labels and settings MUST otherwise stay in effect. + +#### Scenario: A published port +- **WHEN** the compose file publishes `"5432:5432"` for `db` +- **THEN** after `siding up` no container of the stack publishes a port + +#### Scenario: Own networks +- **WHEN** `api` is on the user's networks `backend` and `default` +- **THEN** after `siding up` it is on `backend`, `default` and `siding` + +### Requirement: Up +`siding up` SHALL run `docker compose -p -f -f .siding/override.yml +up -d --remove-orphans`, followed by any arguments after `--`. It SHALL print the stack's +address, with `--json` as a `Stack` document, and fail with `ComposeFailed` and Compose's +message when Compose fails. + +#### Scenario: Passing arguments through +- **WHEN** `siding up -- --build web` runs +- **THEN** Compose builds and starts `web` and its dependencies + +#### Scenario: Compose fails +- **WHEN** an image of the compose file does not exist +- **THEN** `siding up` exits 1 with `ComposeFailed` and Compose's error in the message + +#### Scenario: Run twice +- **WHEN** `siding up` runs again in the same worktree +- **THEN** the same project is updated, and no second stack appears + +### Requirement: Up and wait +`siding up --wait [--timeout ]` SHALL start the stack as `up` does and then end exactly like +`siding status --wait`, with its document and exit code. + +#### Scenario: An agent starts and waits +- **WHEN** an agent runs `siding up --wait --json` and every service becomes ready +- **THEN** it prints one `Status` document with the verdict Ready and exits 0 + +### Requirement: Down +`siding down` SHALL remove the stack's containers and networks by project name, also when the +compose file is gone, and keep its volumes. It SHALL print what it removed, with `--json` as a +`Down` document, and with `--dry-run` remove nothing. + +#### Scenario: Data survives +- **WHEN** `siding down` and then `siding up` run +- **THEN** the stack's named volumes still hold their data + +#### Scenario: Dry run +- **WHEN** `siding down --dry-run --json` runs +- **THEN** it prints a `Down` document listing the containers and networks with `dryRun: true`, and everything still runs + +#### Scenario: Nothing running +- **WHEN** `siding down` runs in a worktree without a stack +- **THEN** it fails with `StackNotFound` + +### Requirement: Removing data keeps work +`siding down --volumes` SHALL also remove the stack's volumes, but MUST refuse with +`UnmergedBranch` when the branch is not merged into the default branch, and with +`DirtyWorktree` when `git status` lists changes, unless `--force` is given. + +#### Scenario: Unmerged branch +- **WHEN** `siding down --volumes` runs on a branch with commits that the default branch lacks +- **THEN** it fails with `UnmergedBranch`, removes nothing, and the hint names `--force` + +#### Scenario: Dirty worktree +- **WHEN** the worktree has uncommitted changes and `siding down --volumes` runs +- **THEN** it fails with `DirtyWorktree` and removes nothing + +#### Scenario: Forced +- **WHEN** `siding down --volumes --force` runs on an unmerged branch +- **THEN** the containers, networks and volumes of the stack are removed + +#### Scenario: No default branch +- **WHEN** siding finds neither `origin/HEAD` nor a local `main` or `master` +- **THEN** the branch counts as unmerged + +### Requirement: Logs +`siding logs [service…]` SHALL show the logs of the stack's services, following them with +`-f` and limited with `--tail `. With `--json` it SHALL print a `Logs` document of the last +lines per service; `--json` together with `-f` MUST fail with `InvalidArguments`. + +#### Scenario: One service +- **WHEN** `siding logs api --tail 5` runs +- **THEN** it prints the last 5 lines of `api` + +### Requirement: No .env values in JSON +Log lines in any document SHALL have every value defined in the worktree's `.env` and in the +services' `env_file`s, of 4 or more characters, replaced by `***`. + +#### Scenario: A password in a log line +- **WHEN** `.env` sets `DB_PASSWORD=not-a-real-token` and a service logs that value +- **THEN** the log line in the `Logs` and `Status` documents shows `***` instead diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-list/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-list/spec.md new file mode 100644 index 0000000..df633c4 --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-list/spec.md @@ -0,0 +1,51 @@ +# Spec Delta + +## Purpose + +Defines `siding ps`, the one view of every stack on the machine: what runs, for which branch +and worktree, at which address, and what memory it takes right now. + +## ADDED Requirements + +### Requirement: Every stack on the machine +`siding ps` SHALL list every stack whose containers carry `dev.siding.repo`, found through the +Docker API from any directory, with repository, branch, state, address, memory in use and +worktree. It needs no git repository and no file of siding's. + +#### Scenario: Two worktrees +- **WHEN** stacks run for `shop` on `main` and on `feature/login` +- **THEN** `siding ps` lists both, each with its own address + +#### Scenario: A stack whose worktree is gone +- **WHEN** a worktree has been deleted while its stack still runs +- **THEN** `siding ps` still lists the stack, and marks the worktree as missing + +#### Scenario: Containers of other projects +- **WHEN** the daemon also runs containers without siding labels +- **THEN** `siding ps` does not list them + +### Requirement: Stack state in the list +The state in `ps` SHALL be `running` when every container of the stack runs, `partial` when +some do, and `stopped` when none does. `ps` MUST NOT probe the proxy; the full verdict is +`siding status`. + +#### Scenario: A stopped container +- **WHEN** one of three containers of a stack has exited +- **THEN** `siding ps` shows the stack as `partial` + +### Requirement: Memory in use +`ps` SHALL show each stack's memory in use, the sum over its running containers as the Docker +API reports it, and the total over all stacks. + +#### Scenario: Two stacks +- **WHEN** one stack uses 300 MiB and another 500 MiB +- **THEN** `siding ps` shows 300 MiB and 500 MiB for them and 800 MiB in total + +### Requirement: StackList document +With `--json`, `ps` SHALL print one `StackList` document with one entry per stack: project, +repository, branch, worktree, machine, agent, state, address, memory in bytes, and +`worktreeMissing` when the worktree is gone. + +#### Scenario: No stacks +- **WHEN** `siding ps --json` runs and no stack exists +- **THEN** it prints a `StackList` document without stacks and exits 0 diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-status/spec.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-status/spec.md new file mode 100644 index 0000000..e375dbd --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/specs/stack-status/spec.md @@ -0,0 +1,85 @@ +# Spec Delta + +## Purpose + +Defines the verdict on a stack that agents and humans wait for: Ready only when it actually +runs and answers through the proxy, otherwise the service that failed, the reason, a hint and +the last lines of its log. + +## ADDED Requirements + +### Requirement: Verdict +`siding status` SHALL report one verdict for the worktree's stack: `Ready`, `Starting`, +`Failed` or `TimedOut`, the state of every service, and the stack's address. Without `--wait` +it SHALL report the current state once. + +#### Scenario: Status without wait +- **WHEN** `siding status` runs while a healthcheck is still starting +- **THEN** it reports `Starting` and exits 2 + +#### Scenario: No stack +- **WHEN** `siding status` runs in a worktree whose project has no containers +- **THEN** it fails with `StackNotFound` and the hint `siding up` + +### Requirement: Ready +A stack SHALL be `Ready` only when every service of the compose file has a container, every +service with a healthcheck is healthy, every other service is running or has exited 0, and +every service with a route path answers through the proxy. + +#### Scenario: Healthy and answering +- **WHEN** `db` is healthy, `web` with `dev.siding.route: "/"` runs and answers through the proxy, and `migrate` has exited 0 +- **THEN** the verdict is `Ready` + +#### Scenario: Running but not answering +- **WHEN** `web` with `dev.siding.route: "/"` runs but its process does not listen yet +- **THEN** the verdict is `Starting`, not `Ready` + +### Requirement: Answering through the proxy +A routed service SHALL count as answering when a request to its route through the proxy gets +an HTTP response from the service itself, of any status; a response the proxy produces on its +own (no router yet, 502, 503, 504) MUST NOT count. + +#### Scenario: The app answers 404 +- **WHEN** `api` with `dev.siding.route: "/api"` answers `/api` with 404 +- **THEN** `api` counts as answering + +#### Scenario: Bad gateway +- **WHEN** the proxy answers 502 for `/api` because nothing listens on the container port +- **THEN** `api` does not count as answering + +### Requirement: Failed +A stack SHALL be `Failed` as soon as a service has exited with a non-zero code, is unhealthy, +keeps restarting, or has no container; the failure MUST name the service, a reason +(`Exited`, `Unhealthy`, `Restarting`, `Missing`, `NotAnswering`), a hint, and the last 20 log +lines of that service. + +#### Scenario: A service exits +- **WHEN** `api` exits with code 1 after printing `cannot connect to db` +- **THEN** the verdict is `Failed` with service `api`, reason `Exited`, the exit code in the hint, and the log tail containing `cannot connect to db` + +#### Scenario: Unhealthy +- **WHEN** `db`'s healthcheck reports unhealthy +- **THEN** the verdict is `Failed` with reason `Unhealthy`, and the hint contains the last healthcheck output + +### Requirement: Waiting +`siding status --wait` SHALL check until the verdict is `Ready` or `Failed`, at most for +`--timeout` (default 3 minutes). When the time is up it SHALL report `TimedOut`, naming the +first service that is not ready, with reason, hint and log tail: `NotAnswering` for a routed +service that never answered, `NotReady` for one that is still starting. + +#### Scenario: Ready in time +- **WHEN** `siding status --wait --timeout 60s` runs and the stack becomes ready after 20 seconds +- **THEN** it reports `Ready` after about 20 seconds and exits 0 + +#### Scenario: Too slow +- **WHEN** `siding status --wait --timeout 10s` runs and `web` with route `/` never answers +- **THEN** it reports `TimedOut` with service `web`, reason `NotAnswering`, and exits 2 + +### Requirement: Status document +With `--json`, `status` SHALL print one `Status` document: the verdict, the stack (project, +repository, branch, worktree, address), every service (state, health, route, address, +answering), and for `Failed` and `TimedOut` the failure with service, reason, hint and log tail. + +#### Scenario: An agent reads the failure +- **WHEN** `siding status --wait --json` ends with `Failed` +- **THEN** the document holds `verdict: Failed` and a `failure` with `service`, `reason`, `hint` and `logs`, and the process exits 1 diff --git a/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/tasks.md b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/tasks.md new file mode 100644 index 0000000..00ae41e --- /dev/null +++ b/openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing/tasks.md @@ -0,0 +1,72 @@ +# Tasks + +## 1. Foundations + +- [x] 1.1 Add `Git`, `Compose`, `Docker`, `Probe`, `Browser`, `Clock`, `Home`, `Env` and `Proxy` to `cli.Options` as interfaces or values (D14), with real implementations wired in `cmd/siding/main.go` and fakes in a test helper; verify `just test` passes and no package-level variable is used as a seam +- [x] 1.2 Add every error code of the spec `cli-contract` with a hint field on `Error`, printed as a second stderr line in text mode and as `hint` in the `Error` document; verify with a table test per code and the updated `error.json` golden +- [x] 1.3 Extend the exit codes with 2 (TimedOut/Starting) and 3 (NoCapacity, unused) and give `Execute` a way to return a verdict's exit code; verify with a unit test of `Execute` + +## 2. Stack identity + +- [x] 2.1 Implement `internal/git`: top level, common dir, current branch (error on detached HEAD), dirty check, merged-into-default-branch check; verify with table tests against temporary repositories (`git init` in `t.TempDir()`), including linked worktrees, detached HEAD, a missing default branch and `.siding/` being ignored +- [x] 2.2 Implement slugs, project name and hosts in `internal/stack`; verify with a table test covering slashes, case, runs of symbols, leading and trailing symbols, 40/41/60 characters and the hash suffix +- [x] 2.3 Implement the machine id (D6) with the os id source and `~/.siding/machine-id` from `Options.Home`; verify that two calls agree, that the raw id never appears, and that the fallback file is created once +- [x] 2.4 Implement the label set (`dev.siding.repo`, `branch`, `worktree`, `machine`, optional `agent` from `--agent` or `SIDING_AGENT`); verify with a unit test + +## 3. Compose config, routing and the override + +- [x] 3.1 Implement `internal/compose`: find the compose file (`--file` or Compose's order), run `config --format json`, parse services, ports, `expose`, labels, networks, `network_mode`, healthchecks, named and external volumes and networks, `env_file`; check Compose >= 2.24; verify with golden inputs captured from the devcontainer's Compose in `internal/compose/testdata/` +- [x] 3.2 Implement HTTP detection and the routing table (service hosts, route paths, `false`, `dev.siding.port`) with the `InvalidRoute` and `AmbiguousPort` errors and the `network_mode` warning; verify with a table test per scenario of the spec `routing` +- [x] 3.3 Generate `.siding/override.yml` (D7) including `ports: !reset []`, networks, Traefik routers, the `X-Siding-Service` headers middleware and labels on own networks and volumes; verify with golden files in `internal/stack/testdata/override/` (`just golden`) and by checking that `docker compose config` accepts each golden override in the devcontainer +- [x] 3.4 Add `dev.siding.port` to the label conventions in CLAUDE.md and `openspec/config.yaml`; verify `openspec validate --all --strict` passes + +## 4. Docker API and the proxy + +- [x] 4.1 Add `github.com/moby/moby/client` and implement `internal/docker`: containers and their state by project label, all siding stacks, published ports of a container, logs tail, one-shot memory stats, network and volume lists by label; `DockerUnavailable` when the daemon is unreachable; verify with unit tests of the mapping from API types to siding's types +- [x] 4.2 Implement `internal/proxy` and `siding setup [--port] [--json]` (D4): embedded compose file, network `siding`, `127.0.0.1:`, the `dev.siding.repo` constraint, idempotence, recreate on port or image change, `PortInUse`; verify with fake-based tests of the decisions and a `setup.json` golden +- [x] 4.3 Document `siding setup` in the README (port 80, `--port`, `--port 8080` in the devcontainer, `*.localhost` resolution); verify the commands in the README match `siding setup --help` + +## 5. Up + +- [x] 5.1 Implement `siding up [--file] [--repo] [--agent] [--json] [-- args]`: identity, `ProxyNotRunning`, `SlugCollision`, write `.siding/override.yml` and `.siding/.gitignore`, run Compose with `--remove-orphans` and the passed-through arguments, `ComposeFailed`, Compose progress on stderr; verify with fake-based tests per scenario of the spec `stack-lifecycle` (Up, Override, never edited) and a `stack.json` golden +- [x] 5.2 Document `siding up` in the README, including that published ports are reset and how to reach a database (`docker compose exec`); verify the example runs as written in the devcontainer + +## 6. Verdict and status + +- [x] 6.1 Implement the probe (D9) against `Options.Probe`: `Host` header, no redirects, `X-Siding-Service` and the 502/503/504 rule; verify with an `httptest` server for each case +- [x] 6.2 Implement `internal/verdict` (D10): Ready, Starting, Failed with `Exited`, `Unhealthy`, `Restarting`, `Missing`, TimedOut with `NotAnswering`, hints, order of evaluation, first failing service in compose order; verify with a table test per scenario of the spec `stack-status` +- [x] 6.3 Implement `siding status [--wait] [--timeout] [--json]` with polling through `Options.Clock`, the 20-line log tail, `StackNotFound`, exit codes 0/1/2; verify with fake-based tests including a timeout without real waiting, and `status-ready.json`, `status-failed.json` and `status-timedout.json` goldens +- [x] 6.4 Add `siding up --wait [--timeout]` ending like `status --wait`; verify with a fake-based test that one `Status` document is printed and the exit code follows the verdict + +## 7. ps + +- [x] 7.1 Implement `siding ps [--json]`: all stacks by `dev.siding.repo`, state `running`/`partial`/`stopped`, address with the proxy port, memory per stack and in total, `worktreeMissing`; verify with fake-based tests and `stacklist.json` and `stacklist-empty.json` goldens +- [x] 7.2 Document `siding ps` and `siding status` in the README; verify the shown output matches the text golden of the tests + +## 8. Logs, open and down + +- [x] 8.1 Implement redaction (D12): parse `.env` and `env_file`s, replace values of 4+ characters, longest first; verify with a table test using `not-a-real-token` and quoted, commented and short values +- [x] 8.2 Implement `siding logs [service…] [-f] [--tail] [--json]` (Compose for text, the Docker API and redaction for the `Logs` document, `InvalidArguments` for `--json -f`); apply redaction to the status log tail too; verify with fake-based tests and a `logs.json` golden +- [x] 8.3 Implement `siding open [service] [--print] [--json]` with `Options.Browser` (`open` on macOS, `$BROWSER` or `xdg-open` on Linux) and the `Address` document; verify with fake-based tests for port 80, 8080 and an unknown service, and an `address.json` golden +- [x] 8.4 Implement `siding down [--volumes] [--force] [--dry-run] [--json]` (D11) by project name, the `UnmergedBranch` and `DirtyWorktree` guard for `--volumes`, `StackNotFound`, the `Down` document; verify with fake-based tests per scenario and `down.json` and `down-dryrun.json` goldens +- [x] 8.5 Make the guardrail "never lose work" precise in CLAUDE.md and `openspec/config.yaml` (data and worktrees are guarded, containers are not) and document `logs`, `open` and `down` in the README; verify `just test` passes + +## 9. Agent skill + +- [x] 9.1 Write the skill text (`internal/cli/skill/SKILL.md`) and `siding skill [--json]` with the `Skill` document; verify with a golden of the printed text, a test that it names `siding up --wait --json`, `status --json`, `logs --json` and exit codes 0/1/2 and none of `--force`, `--volumes`, `share`, and a `skill.json` golden +- [x] 9.2 Document installing the skill in the README; verify the shown command produces the file in a temporary directory + +## 10. End-to-end and CI + +- [x] 10.1 Write `internal/e2e` with `//go:build docker` (D14): proxy `siding-test-proxy` on network `siding-test` at 18080, temporary repositories with `--repo siding-test-…`, a two-service app with `/` and `/api`, cleanup by exact name in `t.Cleanup`; cases: setup idempotent, `up --wait` Ready, Host and path routing, the service host, `X-Siding-Service` present on app responses and the 502 rule, a failing service giving `Failed` with log tail and a redacted `.env` value, `ps`, `down` keeping volumes, `down --volumes` refused when dirty and unmerged and allowed with `--force`; verify with `just smoke-stacks` in the devcontainer +- [x] 10.2 Add the recipe `smoke-stacks` (`hack/smoke/stacks.sh` with `require_devcontainer`, running `go test -tags docker ./internal/e2e`) to the justfile and the CI smoke job, and verify afterwards that no `siding-test-*` object is left in the inner daemon and that `just test` still passes without Docker +- [x] 10.3 Update the dev-environment table in CLAUDE.md and the README with `just smoke-stacks` and `siding setup --port 8080` for the devcontainer; verify the commands run as written +- [x] 10.4 Run the whole flow by hand in the devcontainer: `just build`, `bin/siding setup --port 8080`, `up --wait` in a sample project, open `http://.localhost:8080` from the host browser, `ps`, `down`; record the result in the phase summary +- [x] 10.5 Update the status in CLAUDE.md, the README's pre-alpha note and the status in `openspec/config.yaml` to v0.1; verify `just test` passes +- [x] 10.6 Make the devcontainer refuse a silent port remap (D15): `requireLocalPort` for 8080 in `.devcontainer/devcontainer.json`, the requirement `Devcontainer settings` in the `dev-environment` delta, CLAUDE.md and the README; verify `devcontainer.json` parses and `openspec validate --all --strict` passes + +## Workflow follow-up + +- Stop and show the result; wait for the maintainer's approval of Phase 1. +- `/opsx:archive` merges the deltas into `openspec/specs/`. +- Tag `v0.1.0` after archive, if the maintainer wants a release. diff --git a/openspec/changes/phase-1-stacks-and-routing/proposal.md b/openspec/changes/phase-1-stacks-and-routing/proposal.md deleted file mode 100644 index 751efc9..0000000 --- a/openspec/changes/phase-1-stacks-and-routing/proposal.md +++ /dev/null @@ -1,36 +0,0 @@ -# Proposal - -> Draft. This change holds the roadmap text only and is marked `skip_specs: true`. -> `/opsx:propose` turns it into a full change: it writes the delta specs, `design.md` and -> `tasks.md`, revises this proposal, and removes `skip_specs` from `.openspec.yaml`. - -## Why - -siding exists to answer "does what my agent built actually run?". v0.1 is the smallest siding -that answers it: one stack per worktree, reachable by name, with a verdict an agent can wait for. - -## What Changes - -Roadmap v0.1: - -- `siding setup`: the shared proxy `siding-proxy` (Traefik, Docker provider) on the network - `siding`, and the stable machine id. -- `siding up`: writes `.siding/override.yml` (ports reset, network `siding`, Traefik and - `dev.siding.*` labels) and runs the user's compose file as project `-`. -- `siding ps`: every stack on the machine, read from labels and the Docker API. -- `siding status --wait --json`: the verdict Ready, or Failed with service, reason, hint and - the last log lines; exit 0 Ready, 1 Failed, 2 TimedOut. -- `siding logs`, `siding open`, `siding down` (refuses unmerged or dirty work unless forced; - `--dry-run`). -- Hostname and path routing: `..localhost`, `dev.siding.route` (`/`, - `/`, `false`), `...localhost` for unrouted HTTP services. -- A Claude Code skill that uses the `--json` documents. - -## Capabilities - -To be decided at `/opsx:propose`. - -## Impact - -Go code under `internal/`, the Docker Go SDK as a dependency, new smoke tests in the -devcontainer's inner daemon. diff --git a/openspec/config.yaml b/openspec/config.yaml index 8ad4033..7497aa7 100644 --- a/openspec/config.yaml +++ b/openspec/config.yaml @@ -21,8 +21,8 @@ context: | - Humans see what runs, what it costs and what can go after a merge. - A memory budget keeps the machine usable; later, stacks can run on a remote Docker host. - Status: pre-alpha. Phase 0 (repository, devcontainer, OpenSpec, CI) is done; `siding version` - is the only command. OpenSpec is the only planning source: `openspec/specs/` is what siding + Status: pre-alpha. Phase 0 (repository, devcontainer, OpenSpec, CI) is done; Phase 1 (v0.1: + stacks per worktree, routing through siding-proxy, the verdict of status) is implemented. OpenSpec is the only planning source: `openspec/specs/` is what siding does today, `openspec/changes/` what it is about to do (the roadmap is the `phase--` changes), `openspec/changes/archive/` how it got here. CLAUDE.md holds the working agreements, guardrails, safety rules and conventions in full; read it before @@ -60,8 +60,10 @@ context: | `apiVersion: siding.dev/v1alpha1` and a `kind`; within a version fields are only added; empty fields are omitted, never null; a failure prints `kind: Error` with a stable `code` and a `message`. Exit codes: 0 Ready, 1 Failed, 2 TimedOut, 3 NoCapacity. - 6. Never lose work. Removing a stack or worktree refuses when the branch is unmerged or the - worktree is dirty, unless forced; every destructive command supports `--dry-run`. + 6. Never lose work. Removing a stack's data (its volumes) or a worktree refuses when the + branch is unmerged or the worktree is dirty, unless forced; removing containers and + networks loses nothing and is not guarded (`siding down` keeps volumes, `down --volumes` + is guarded). Every destructive command supports `--dry-run`. 7. Only a human exposes anything to the internet. `siding share` is interactive only and has no `--yes`. 8. Standard building blocks (Docker Compose, Traefik, later Sablier) over custom code. @@ -80,8 +82,11 @@ context: | every stack by Host and path; no stack publishes a port. - Labels on every container, network and volume siding creates or adopts: `dev.siding.repo`, `dev.siding.branch`, `dev.siding.worktree` (absolute path), `dev.siding.machine` (stable - machine id), `dev.siding.agent` (optional). Service labels: `dev.siding.route`, - `dev.siding.seed` (`volume|script|none`). Volume label `dev.siding.shared`. The machine and + machine id); on containers also `dev.siding.agent` (optional; network and volume labels + must not change between two `up`s, since Compose recreates a network whose labels changed). + Service labels: `dev.siding.route` (`/`, `/`, `false`), `dev.siding.port` (the HTTP + port of a service with several), `dev.siding.seed` (`volume|script|none`). Volume label + `dev.siding.shared`. The machine and worktree labels exist from v0.1 so that remote hosts need no migration later. - Development and every test run in a devcontainer with its own Docker daemon (Docker-in-Docker); see the spec `dev-environment`. diff --git a/openspec/specs/agent-skill/spec.md b/openspec/specs/agent-skill/spec.md new file mode 100644 index 0000000..2ecde2d --- /dev/null +++ b/openspec/specs/agent-skill/spec.md @@ -0,0 +1,25 @@ +# agent-skill Specification + +## Purpose +Defines the Claude Code skill siding ships, so that a coding agent starts its worktree's stack +and waits for the verdict instead of guessing whether its work runs. + +## Requirements + +### Requirement: siding skill prints the skill +`siding skill` SHALL print a Claude Code skill (`SKILL.md` with name, description and body) on +stdout and write no file. With `--json` it SHALL print a `Skill` document holding the same text. + +#### Scenario: Install the skill +- **WHEN** a user runs `mkdir -p .claude/skills/siding && siding skill > .claude/skills/siding/SKILL.md` +- **THEN** the file is a valid skill named `siding`, and siding wrote nothing itself + +### Requirement: What the skill teaches +The skill SHALL tell the agent to run `siding up --wait --json` after a change that affects the +running app, to read the verdict and exit code, to fix the named service from the failure's +reason, hint and log tail, and to report the address when Ready. It MUST NOT tell the agent to +use `--force`, `--volumes` or `siding share`. + +#### Scenario: The commands in the skill +- **WHEN** the printed skill is read +- **THEN** it names `siding up --wait --json`, `siding status --json`, `siding logs --json` and the exit codes 0, 1 and 2, and none of `--force`, `--volumes` or `share` diff --git a/openspec/specs/cli-contract/spec.md b/openspec/specs/cli-contract/spec.md new file mode 100644 index 0000000..0bdd6ed --- /dev/null +++ b/openspec/specs/cli-contract/spec.md @@ -0,0 +1,75 @@ +# cli-contract Specification + +## Purpose +Defines what agents and scripts can rely on when they call siding: one versioned JSON document +per command with `--json`, stable error codes and fixed exit codes. + +## Requirements + +### Requirement: One JSON document per command +Every siding command SHALL accept `--json` and then print exactly one JSON document on stdout +and nothing else there. The document SHALL carry `apiVersion: siding.dev/v1alpha1` and a +`kind`. Empty fields MUST be left out, never printed as `null`. + +#### Scenario: A command with --json +- **WHEN** `siding version --json` runs +- **THEN** stdout holds one document with `apiVersion` `siding.dev/v1alpha1` and `kind` `Version`, and nothing else + +#### Scenario: Progress of docker compose +- **WHEN** a command with `--json` runs `docker compose`, which reports progress +- **THEN** that progress goes to stderr, and stdout still holds only the document + +#### Scenario: An empty field +- **WHEN** a document has a field without a value, such as a commit unknown to the build +- **THEN** the field is missing from the document + +### Requirement: Compatible within a version +Within `siding.dev/v1alpha1` a document field and an error code MUST NOT be removed, renamed +or given another meaning; new fields, kinds and codes MAY be added. + +#### Scenario: A field is added +- **WHEN** a later release adds a field to the `Status` document +- **THEN** documents still carry `apiVersion` `siding.dev/v1alpha1` and every earlier field with its meaning + +### Requirement: Error document +A command that fails SHALL exit 1. With `--json` it SHALL print a document of kind `Error` +with a stable `code` and a human-readable `message`; without `--json` it SHALL print +`error: ` on stderr, followed by a hint line when there is one. + +#### Scenario: An unknown flag with --json +- **WHEN** `siding version --json --nope` runs +- **THEN** it exits 1 and prints an `Error` document with `code` `InvalidArguments`, even though flag parsing failed before `--json` was read + +#### Scenario: A failure without --json +- **WHEN** `siding up` runs outside a git repository +- **THEN** it exits 1 and prints `error: …` and a hint on stderr, and nothing on stdout + +#### Scenario: The error codes +- **WHEN** a command fails +- **THEN** its code is one of `InvalidArguments`, `NotAGitRepository`, `NoBranch`, `ComposeFileNotFound`, `ComposeFailed`, `DockerUnavailable`, `ProxyNotRunning`, `PortInUse`, `InvalidRoute`, `AmbiguousPort`, `SlugCollision`, `StackNotFound`, `UnmergedBranch`, `DirtyWorktree`, or `Internal` for an unexpected error + +### Requirement: Exit codes +siding SHALL exit 0 on success and when a stack is Ready, 1 when a command fails or a stack is +Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`), 3 for +NoCapacity, which v0.1 never returns, and 130 without a message when interrupted. + +#### Scenario: A verdict decides the exit code +- **WHEN** `siding status --wait` ends with the verdict Failed +- **THEN** it exits 1, and with Ready it exits 0 + +#### Scenario: Interrupted +- **WHEN** the user presses Ctrl-C during `siding logs -f` +- **THEN** siding stops docker compose and exits 130 without an error message + +### Requirement: Version +`siding version` SHALL print the version, the commit (when the build knows it), the Go version +and the platform; with `--json` as a document of kind `Version` with the fields `version`, +`commit`, `goVersion` and `platform`. + +#### Scenario: Text output +- **WHEN** `siding version` runs in a build of commit `0123456789abcdef…` +- **THEN** it prints one line such as `siding v0.1.0 (0123456789ab) go1.27.3 linux/arm64` + +#### Scenario: An argument +- **WHEN** `siding version extra` runs +- **THEN** it fails with `InvalidArguments` diff --git a/openspec/specs/dev-environment/spec.md b/openspec/specs/dev-environment/spec.md index aada57d..5eda724 100644 --- a/openspec/specs/dev-environment/spec.md +++ b/openspec/specs/dev-environment/spec.md @@ -32,8 +32,9 @@ stack siding's tests create, and MUST NOT mount or reach the host's Docker socke ### Requirement: Devcontainer settings The devcontainer SHALL be named `siding-devcontainer`, set `SIDING_DEVCONTAINER=1`, forward port -8080 inside to 8080 on the host for the proxy, and set `git config worktree.useRelativePaths true` -when it is created, so worktrees work both inside the container and on the host. +8080 inside to 8080 on the host for the proxy and never to another host port, and set +`git config worktree.useRelativePaths true` when it is created, so worktrees work both inside +the container and on the host. #### Scenario: A worktree made in the container - **WHEN** `git worktree add` runs inside the devcontainer @@ -43,6 +44,10 @@ when it is created, so worktrees work both inside the container and on the host. - **WHEN** a worktree with relative paths exists, and Git before 2.48 on the host opens the repository - **THEN** it refuses with "unknown repository extension found: relativeworktrees", so the host needs Git 2.48 or later +#### Scenario: Port 8080 is taken on the host +- **WHEN** the devcontainer opens while another process holds port 8080 on the host +- **THEN** VS Code reports that it cannot forward 8080, instead of forwarding to another port that the addresses siding prints do not name + ### Requirement: Guard in the hack scripts Every script in `hack/` except `hack/nuke.sh` MUST abort unless `SIDING_DEVCONTAINER=1` and the Docker daemon reports the container's own host name. @@ -86,6 +91,24 @@ port the compose file publishes, and that Traefik with the Docker provider on th - **WHEN** the demo project runs with the override - **THEN** none of its containers has a published port, although the compose file publishes one +### Requirement: Stack smoke test +`just smoke-stacks` SHALL run siding's own commands end to end against the inner daemon: +`setup`, `up --wait`, routing by host and path, a `Failed` verdict with log tail, `ps`, `down` +and the refusal of `down --volumes`. Its proxy, network and projects MUST carry the prefix +`siding-test-`, and its proxy MUST listen on 18080. + +#### Scenario: Stack smoke test in CI +- **WHEN** a pull request is opened or pushed to, or a commit is pushed to `main` +- **THEN** CI runs `just smoke-stacks` inside the devcontainer, against its own daemon + +#### Scenario: It leaves nothing behind +- **WHEN** `just smoke-stacks` ends, passing or failing +- **THEN** no container, network or volume whose name starts with `siding-test-` is left in the inner daemon, and `siding-proxy` and the network `siding` are untouched + +#### Scenario: Not part of level 1 +- **WHEN** `just test` runs without a Docker daemon +- **THEN** it passes, because the end-to-end tests only build with the `docker` build tag + ### Requirement: Test stacks Tests SHALL create stacks only as Compose projects whose name starts with `siding-test-`, and MUST remove them by their exact project name, also when the test fails. The smoke test's proxy diff --git a/openspec/specs/proxy/spec.md b/openspec/specs/proxy/spec.md new file mode 100644 index 0000000..8a142cc --- /dev/null +++ b/openspec/specs/proxy/spec.md @@ -0,0 +1,66 @@ +# proxy Specification + +## Purpose +Defines the one shared proxy through which every stack is reached: `siding-proxy`, a Traefik +on the network `siding`, started by `siding setup` and bound to the loopback interface only. + +## Requirements + +### Requirement: Setup starts the proxy +`siding setup` SHALL create the network `siding` if it is missing and run the Compose project +`siding` with one container `siding-proxy`: Traefik with the Docker provider on the network +`siding`, restarting unless stopped. It SHALL print the proxy's address, with `--json` as a +`Setup` document. + +#### Scenario: First setup +- **WHEN** `siding setup` runs on a machine without the proxy +- **THEN** the network `siding` and the running container `siding-proxy` exist, and it prints `http://*.localhost` as the address + +#### Scenario: JSON +- **WHEN** `siding setup --json` runs +- **THEN** it prints a `Setup` document with the proxy's container name, network, image and port + +### Requirement: Loopback port +The proxy SHALL publish exactly one port, on `127.0.0.1` only, `80` unless `--port ` names +another. siding MUST read the proxy's port from the running container, not from a file. + +#### Scenario: Default port +- **WHEN** `siding setup` has run +- **THEN** `siding-proxy` publishes `127.0.0.1:80` and no other port, and nothing listens for it on other interfaces + +#### Scenario: Another port +- **WHEN** `siding setup --port 8080` has run and `siding open` runs later +- **THEN** the address it opens is `http://:8080` + +#### Scenario: The port is taken +- **WHEN** another process holds `127.0.0.1:80` and `siding setup` runs +- **THEN** it fails with `PortInUse` and a hint to use `--port` + +### Requirement: Setup is idempotent +`siding setup` SHALL change nothing when the proxy already runs as requested, and SHALL +recreate only the proxy container when its port or image differs. Running stacks MUST keep +running and be routed again once the proxy is back. + +#### Scenario: Second run +- **WHEN** `siding setup` runs twice +- **THEN** the second run reports the proxy as already running and recreates nothing + +#### Scenario: Port change with running stacks +- **WHEN** a stack runs and `siding setup --port 8080` follows `siding setup` +- **THEN** the stack's containers are not restarted and it answers on port 8080 + +### Requirement: Only siding stacks are routed +The proxy MUST route only containers that carry `dev.siding.repo` and enable Traefik, so that +other containers with Traefik labels on the same daemon are never exposed through it. + +#### Scenario: A foreign Traefik label +- **WHEN** a container without siding labels carries `traefik.enable=true` and a Host rule +- **THEN** the proxy answers 404 for that host + +### Requirement: Stacks need the proxy +`up` and `status` SHALL fail with `ProxyNotRunning` and the hint `siding setup` when +`siding-proxy` is not running. + +#### Scenario: up before setup +- **WHEN** `siding up` runs on a machine without the proxy +- **THEN** it fails with `ProxyNotRunning` before starting any container diff --git a/openspec/specs/routing/spec.md b/openspec/specs/routing/spec.md new file mode 100644 index 0000000..1dc1921 --- /dev/null +++ b/openspec/specs/routing/spec.md @@ -0,0 +1,88 @@ +# routing Specification + +## Purpose +Defines how the services of a stack are reached through the proxy: which services count as +HTTP services, how the label `dev.siding.route` maps them to the stack's host and paths, and +how `siding open` finds the address. + +## Requirements + +### Requirement: HTTP services +A service SHALL be an HTTP service when the user's compose file gives it exactly one distinct +container port, from the targets of `ports` and from `expose`, or when its label +`dev.siding.port` names one. A service with several ports and no `dev.siding.port` is not one. + +#### Scenario: One published port +- **WHEN** a service publishes `"3000:3000"` +- **THEN** it is an HTTP service on container port 3000, and the port is no longer published + +#### Scenario: Several ports with the label +- **WHEN** a service exposes 3000 and 9229 and carries `dev.siding.port: "3000"` +- **THEN** it is an HTTP service on container port 3000 + +#### Scenario: No port +- **WHEN** a worker service has neither `ports` nor `expose` nor `dev.siding.port` +- **THEN** it is not routed + +### Requirement: Service host +Every HTTP service SHALL answer at `...localhost`, unless it +carries `dev.siding.route: "false"`. + +#### Scenario: An unlabelled HTTP service +- **WHEN** the service `mail` exposes 8025 and has no route label, in the stack `shop` on branch `main` +- **THEN** `mail.shop.main.localhost` reaches it on 8025 + +#### Scenario: Hidden +- **WHEN** a service carries `dev.siding.route: "false"` +- **THEN** no host or path reaches it through the proxy + +### Requirement: Route label +`dev.siding.route: "/"` SHALL make an HTTP service the root of the stack's host, and +`dev.siding.route: "/"` SHALL mount it on the stack's host under that path prefix, +passing the path on unchanged. The longest matching prefix MUST win. + +#### Scenario: Root and API +- **WHEN** `web` carries `dev.siding.route: "/"` and `api` carries `dev.siding.route: "/api"` in the stack `shop` on `main` +- **THEN** `shop.main.localhost/` reaches `web`, and `shop.main.localhost/api/users` reaches `api` with the path `/api/users` + +#### Scenario: No root +- **WHEN** no service carries `dev.siding.route: "/"` +- **THEN** a path on the stack's host that no route covers answers 404 + +### Requirement: Invalid routes +`siding up` SHALL fail with `InvalidRoute`, naming the service, and start nothing when a route +label is neither `false` nor an absolute path, when two services claim the same path, or when +a service with a route is not an HTTP service. + +#### Scenario: Two services on one path +- **WHEN** `web` and `admin` both carry `dev.siding.route: "/"` +- **THEN** `siding up` fails with `InvalidRoute` naming both + +#### Scenario: A route without a port +- **WHEN** a service carries `dev.siding.route: "/api"` but has no container port +- **THEN** `siding up` fails with `InvalidRoute` and a hint to add `expose` or `dev.siding.port` + +#### Scenario: A route with several ports +- **WHEN** a service carries `dev.siding.route: "/"` and exposes 3000 and 9229 without `dev.siding.port` +- **THEN** `siding up` fails with `AmbiguousPort` naming the ports + +### Requirement: Services that cannot join the proxy network +A service with a `network_mode` SHALL NOT be routed; `up` SHALL warn about it on stderr, and +fail with `InvalidRoute` when it carries a route path. + +#### Scenario: network_mode host +- **WHEN** a service uses `network_mode: host` and has no route label +- **THEN** `siding up` starts it unrouted and prints a warning + +### Requirement: Open +`siding open [service]` SHALL open the stack's address, or the service's host, in the default +browser; with `--print` or `--json` it SHALL only print it, as text or as an `Address` +document. The address MUST include the proxy's port unless it is 80. + +#### Scenario: Print the address +- **WHEN** the proxy listens on 8080 and `siding open --print` runs in the stack `shop` on `main` +- **THEN** it prints `http://shop.main.localhost:8080/` + +#### Scenario: Unknown service +- **WHEN** `siding open nope` runs and the stack has no HTTP service `nope` +- **THEN** it fails with `InvalidArguments` and lists the HTTP services diff --git a/openspec/specs/stack-identity/spec.md b/openspec/specs/stack-identity/spec.md new file mode 100644 index 0000000..d831c60 --- /dev/null +++ b/openspec/specs/stack-identity/spec.md @@ -0,0 +1,100 @@ +# stack-identity Specification + +## Purpose +Defines how siding derives a stack's identity from a git worktree: repository name, branch +slug, Compose project, host names, machine id and the labels that let Docker answer every +later question about the stack. + +## Requirements + +### Requirement: One stack per worktree +siding SHALL treat the git worktree containing the current directory as the stack's worktree, +identified by its absolute top-level path. Every command that acts on a stack MUST act on that +worktree's stack. + +#### Scenario: Run from a subdirectory +- **WHEN** `siding status` runs in `/src/shop-login/web/src`, inside the worktree `/src/shop-login` +- **THEN** it reports the stack of `/src/shop-login` + +#### Scenario: Outside a repository +- **WHEN** a stack command runs outside any git worktree +- **THEN** it fails with `NotAGitRepository` + +### Requirement: Repository name +The repository name SHALL be the name of the directory that holds the repository's common git +directory (the main worktree), the same for every worktree of the repository. `--repo ` +MUST override it. + +#### Scenario: Two worktrees of one repository +- **WHEN** `/src/shop` is the main worktree and `/src/shop-login` a linked worktree of it +- **THEN** both have the repository name `shop` + +#### Scenario: Override +- **WHEN** `siding up --repo store` runs in `/src/shop` +- **THEN** the repository name is `store` + +### Requirement: Slugs +The repository name and the branch SHALL each be turned into a slug: lowercase, every run of +characters other than `a-z` and `0-9` replaced by one `-`, no leading or trailing `-`. A slug +longer than 40 characters MUST be cut to 33 characters, followed by `-` and the first 6 hex +digits of the SHA-256 of the original name. + +#### Scenario: A branch with a slash +- **WHEN** the branch is `feature/Login_Form` +- **THEN** its slug is `feature-login-form` + +#### Scenario: A long branch +- **WHEN** the branch name is 60 characters long +- **THEN** its slug is 40 characters long and ends in `-` and 6 hex digits that differ for another branch with the same first 33 characters + +### Requirement: Branch required +The worktree SHALL be on a branch. In a detached HEAD a stack command MUST fail with `NoBranch` +and a hint to check out a branch. + +#### Scenario: Detached HEAD +- **WHEN** `siding up` runs in a worktree whose HEAD is detached +- **THEN** it fails with `NoBranch` and starts nothing + +### Requirement: Project name and host +The Compose project SHALL be named `-` and the stack's host SHALL be +`..localhost`. + +#### Scenario: Names +- **WHEN** the repository is `shop` and the branch `feature/login` +- **THEN** the project is `shop-feature-login` and the host `shop.feature-login.localhost` + +### Requirement: Slug collision +siding MUST NOT let two worktrees or branches share one project. If the project already exists +with another `dev.siding.worktree` or `dev.siding.branch` label, `up` MUST fail with +`SlugCollision`, naming the other worktree and branch, and change nothing. + +#### Scenario: feature/a and feature-a +- **WHEN** a stack for branch `feature/a` runs and `siding up` runs in a worktree on branch `feature-a` +- **THEN** it fails with `SlugCollision` and the running stack is untouched + +### Requirement: Labels +siding SHALL label every container, network and volume of a stack with `dev.siding.repo`, +`dev.siding.branch` (the branch name, not the slug), `dev.siding.worktree` (absolute path) and +`dev.siding.machine`, and its containers with `dev.siding.agent` when `--agent` or +`SIDING_AGENT` names one. External networks and volumes MUST NOT be relabelled. + +#### Scenario: Labels on a running stack +- **WHEN** `siding up` has run in `/src/shop-login` on branch `feature/login` +- **THEN** every container, the project's networks and its named volumes carry `dev.siding.repo=shop`, `dev.siding.branch=feature/login` and `dev.siding.worktree=/src/shop-login` + +#### Scenario: An agent name +- **WHEN** `SIDING_AGENT=claude siding up` runs +- **THEN** the containers also carry `dev.siding.agent=claude` + +### Requirement: Machine id +`dev.siding.machine` SHALL be a stable id of the machine siding runs on: derived from the +operating system's machine id by a one-way hash, never the raw id. Without an operating system +id, siding MUST generate one and keep it in `~/.siding/machine-id`. + +#### Scenario: Two runs +- **WHEN** `siding up` runs twice on the same machine +- **THEN** both stacks carry the same `dev.siding.machine` + +#### Scenario: No operating system id +- **WHEN** neither `/etc/machine-id` nor the macOS platform UUID is available +- **THEN** siding writes a random id to `~/.siding/machine-id` and uses it from then on diff --git a/openspec/specs/stack-lifecycle/spec.md b/openspec/specs/stack-lifecycle/spec.md new file mode 100644 index 0000000..3de30f3 --- /dev/null +++ b/openspec/specs/stack-lifecycle/spec.md @@ -0,0 +1,124 @@ +# stack-lifecycle Specification + +## Purpose +Defines how a worktree's stack is started, inspected through its logs and taken down: the +user's compose file stays untouched, siding adds one override, and nothing removes unmerged or +uncommitted work without being forced. + +## Requirements + +### Requirement: Compose file +siding SHALL use the compose file Compose itself would pick in the worktree's top-level +directory (`compose.yaml`, `compose.yml`, `docker-compose.yaml`, `docker-compose.yml`), or the +one named by `--file`. Without one, stack commands MUST fail with `ComposeFileNotFound`. + +#### Scenario: The default file +- **WHEN** the worktree holds `compose.yaml` +- **THEN** `siding up` uses it + +#### Scenario: No compose file +- **WHEN** the worktree holds no compose file and `--file` is not given +- **THEN** `siding up` fails with `ComposeFileNotFound` + +### Requirement: The compose file is never edited +siding MUST NOT write to the user's compose file, its `.env` or any other file of the user. It +SHALL write only `.siding/override.yml` and `.siding/.gitignore` (containing `*`) in the +worktree, so that `.siding/` never makes the worktree dirty. + +#### Scenario: After up +- **WHEN** `siding up` has run in a clean worktree +- **THEN** `git status --porcelain` prints nothing and the compose file is byte for byte unchanged + +### Requirement: Override +The override SHALL reset every service's `ports` (`ports: !reset []`), add every routed +service to the network `siding` next to its own networks, and add the `dev.siding.*` and +Traefik labels. The user's labels and settings MUST otherwise stay in effect. + +#### Scenario: A published port +- **WHEN** the compose file publishes `"5432:5432"` for `db` +- **THEN** after `siding up` no container of the stack publishes a port + +#### Scenario: Own networks +- **WHEN** `api` is on the user's networks `backend` and `default` +- **THEN** after `siding up` it is on `backend`, `default` and `siding` + +### Requirement: Up +`siding up` SHALL run `docker compose -p -f -f .siding/override.yml +up -d --remove-orphans`, followed by any arguments after `--`. It SHALL print the stack's +address, with `--json` as a `Stack` document, and fail with `ComposeFailed` and Compose's +message when Compose fails. + +#### Scenario: Passing arguments through +- **WHEN** `siding up -- --build web` runs +- **THEN** Compose builds and starts `web` and its dependencies + +#### Scenario: Compose fails +- **WHEN** an image of the compose file does not exist +- **THEN** `siding up` exits 1 with `ComposeFailed` and Compose's error in the message + +#### Scenario: Run twice +- **WHEN** `siding up` runs again in the same worktree +- **THEN** the same project is updated, and no second stack appears + +### Requirement: Up and wait +`siding up --wait [--timeout ]` SHALL start the stack as `up` does and then end exactly like +`siding status --wait`, with its document and exit code. + +#### Scenario: An agent starts and waits +- **WHEN** an agent runs `siding up --wait --json` and every service becomes ready +- **THEN** it prints one `Status` document with the verdict Ready and exits 0 + +### Requirement: Down +`siding down` SHALL remove the stack's containers and networks by project name, also when the +compose file is gone, and keep its volumes. It SHALL print what it removed, with `--json` as a +`Down` document, and with `--dry-run` remove nothing. + +#### Scenario: Data survives +- **WHEN** `siding down` and then `siding up` run +- **THEN** the stack's named volumes still hold their data + +#### Scenario: Dry run +- **WHEN** `siding down --dry-run --json` runs +- **THEN** it prints a `Down` document listing the containers and networks with `dryRun: true`, and everything still runs + +#### Scenario: Nothing running +- **WHEN** `siding down` runs in a worktree without a stack +- **THEN** it fails with `StackNotFound` + +### Requirement: Removing data keeps work +`siding down --volumes` SHALL also remove the stack's volumes, but MUST refuse with +`UnmergedBranch` when the branch is not merged into the default branch, and with +`DirtyWorktree` when `git status` lists changes, unless `--force` is given. + +#### Scenario: Unmerged branch +- **WHEN** `siding down --volumes` runs on a branch with commits that the default branch lacks +- **THEN** it fails with `UnmergedBranch`, removes nothing, and the hint names `--force` + +#### Scenario: Dirty worktree +- **WHEN** the worktree has uncommitted changes and `siding down --volumes` runs +- **THEN** it fails with `DirtyWorktree` and removes nothing + +#### Scenario: Forced +- **WHEN** `siding down --volumes --force` runs on an unmerged branch +- **THEN** the containers, networks and volumes of the stack are removed + +#### Scenario: No default branch +- **WHEN** siding finds neither `origin/HEAD` nor a local `main` or `master` +- **THEN** the branch counts as unmerged + +### Requirement: Logs +`siding logs [service…]` SHALL show the logs of the stack's services, following them with +`-f` and limited with `--tail `. With `--json` it SHALL print a `Logs` document of the last +lines per service; `--json` together with `-f` MUST fail with `InvalidArguments`. + +#### Scenario: One service +- **WHEN** `siding logs api --tail 5` runs +- **THEN** it prints the last 5 lines of `api` + +### Requirement: No .env values in JSON +Log lines in any document SHALL have every value defined in the worktree's `.env` and in the +services' `env_file`s, of 4 or more characters, replaced by `***`. + +#### Scenario: A password in a log line +- **WHEN** `.env` sets `DB_PASSWORD=not-a-real-token` and a service logs that value +- **THEN** the log line in the `Logs` and `Status` documents shows `***` instead diff --git a/openspec/specs/stack-list/spec.md b/openspec/specs/stack-list/spec.md new file mode 100644 index 0000000..3a7b27b --- /dev/null +++ b/openspec/specs/stack-list/spec.md @@ -0,0 +1,50 @@ +# stack-list Specification + +## Purpose +Defines `siding ps`, the one view of every stack on the machine: what runs, for which branch +and worktree, at which address, and what memory it takes right now. + +## Requirements + +### Requirement: Every stack on the machine +`siding ps` SHALL list every stack whose containers carry `dev.siding.repo`, found through the +Docker API from any directory, with repository, branch, state, address, memory in use and +worktree. It needs no git repository and no file of siding's. + +#### Scenario: Two worktrees +- **WHEN** stacks run for `shop` on `main` and on `feature/login` +- **THEN** `siding ps` lists both, each with its own address + +#### Scenario: A stack whose worktree is gone +- **WHEN** a worktree has been deleted while its stack still runs +- **THEN** `siding ps` still lists the stack, and marks the worktree as missing + +#### Scenario: Containers of other projects +- **WHEN** the daemon also runs containers without siding labels +- **THEN** `siding ps` does not list them + +### Requirement: Stack state in the list +The state in `ps` SHALL be `running` when every container of the stack runs, `partial` when +some do, and `stopped` when none does. `ps` MUST NOT probe the proxy; the full verdict is +`siding status`. + +#### Scenario: A stopped container +- **WHEN** one of three containers of a stack has exited +- **THEN** `siding ps` shows the stack as `partial` + +### Requirement: Memory in use +`ps` SHALL show each stack's memory in use, the sum over its running containers as the Docker +API reports it, and the total over all stacks. + +#### Scenario: Two stacks +- **WHEN** one stack uses 300 MiB and another 500 MiB +- **THEN** `siding ps` shows 300 MiB and 500 MiB for them and 800 MiB in total + +### Requirement: StackList document +With `--json`, `ps` SHALL print one `StackList` document with one entry per stack: project, +repository, branch, worktree, machine, agent, state, address, memory in bytes, and +`worktreeMissing` when the worktree is gone. + +#### Scenario: No stacks +- **WHEN** `siding ps --json` runs and no stack exists +- **THEN** it prints a `StackList` document without stacks and exits 0 diff --git a/openspec/specs/stack-status/spec.md b/openspec/specs/stack-status/spec.md new file mode 100644 index 0000000..b069b37 --- /dev/null +++ b/openspec/specs/stack-status/spec.md @@ -0,0 +1,84 @@ +# stack-status Specification + +## Purpose +Defines the verdict on a stack that agents and humans wait for: Ready only when it actually +runs and answers through the proxy, otherwise the service that failed, the reason, a hint and +the last lines of its log. + +## Requirements + +### Requirement: Verdict +`siding status` SHALL report one verdict for the worktree's stack: `Ready`, `Starting`, +`Failed` or `TimedOut`, the state of every service, and the stack's address. Without `--wait` +it SHALL report the current state once. + +#### Scenario: Status without wait +- **WHEN** `siding status` runs while a healthcheck is still starting +- **THEN** it reports `Starting` and exits 2 + +#### Scenario: No stack +- **WHEN** `siding status` runs in a worktree whose project has no containers +- **THEN** it fails with `StackNotFound` and the hint `siding up` + +### Requirement: Ready +A stack SHALL be `Ready` only when every service of the compose file has a container, every +service with a healthcheck is healthy, every other service is running or has exited 0, and +every service with a route path answers through the proxy. + +#### Scenario: Healthy and answering +- **WHEN** `db` is healthy, `web` with `dev.siding.route: "/"` runs and answers through the proxy, and `migrate` has exited 0 +- **THEN** the verdict is `Ready` + +#### Scenario: Running but not answering +- **WHEN** `web` with `dev.siding.route: "/"` runs but its process does not listen yet +- **THEN** the verdict is `Starting`, not `Ready` + +### Requirement: Answering through the proxy +A routed service SHALL count as answering when a request to its route through the proxy gets +an HTTP response from the service itself, of any status; a response the proxy produces on its +own (no router yet, 502, 503, 504) MUST NOT count. + +#### Scenario: The app answers 404 +- **WHEN** `api` with `dev.siding.route: "/api"` answers `/api` with 404 +- **THEN** `api` counts as answering + +#### Scenario: Bad gateway +- **WHEN** the proxy answers 502 for `/api` because nothing listens on the container port +- **THEN** `api` does not count as answering + +### Requirement: Failed +A stack SHALL be `Failed` as soon as a service has exited with a non-zero code, is unhealthy, +keeps restarting, or has no container; the failure MUST name the service, a reason +(`Exited`, `Unhealthy`, `Restarting`, `Missing`, `NotAnswering`), a hint, and the last 20 log +lines of that service. + +#### Scenario: A service exits +- **WHEN** `api` exits with code 1 after printing `cannot connect to db` +- **THEN** the verdict is `Failed` with service `api`, reason `Exited`, the exit code in the hint, and the log tail containing `cannot connect to db` + +#### Scenario: Unhealthy +- **WHEN** `db`'s healthcheck reports unhealthy +- **THEN** the verdict is `Failed` with reason `Unhealthy`, and the hint contains the last healthcheck output + +### Requirement: Waiting +`siding status --wait` SHALL check until the verdict is `Ready` or `Failed`, at most for +`--timeout` (default 3 minutes). When the time is up it SHALL report `TimedOut`, naming the +first service that is not ready, with reason, hint and log tail: `NotAnswering` for a routed +service that never answered, `NotReady` for one that is still starting. + +#### Scenario: Ready in time +- **WHEN** `siding status --wait --timeout 60s` runs and the stack becomes ready after 20 seconds +- **THEN** it reports `Ready` after about 20 seconds and exits 0 + +#### Scenario: Too slow +- **WHEN** `siding status --wait --timeout 10s` runs and `web` with route `/` never answers +- **THEN** it reports `TimedOut` with service `web`, reason `NotAnswering`, and exits 2 + +### Requirement: Status document +With `--json`, `status` SHALL print one `Status` document: the verdict, the stack (project, +repository, branch, worktree, address), every service (state, health, route, address, +answering), and for `Failed` and `TimedOut` the failure with service, reason, hint and log tail. + +#### Scenario: An agent reads the failure +- **WHEN** `siding status --wait --json` ends with `Failed` +- **THEN** the document holds `verdict: Failed` and a `failure` with `service`, `reason`, `hint` and `logs`, and the process exits 1