diff --git a/CLAUDE.md b/CLAUDE.md index 5f56d7f..9f827cd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,12 +17,14 @@ changing architecture. Phase 0 approved (2026-10-28). Phase 1 (v0.1) and Phase 2 (v0.2) approved and archived (2026-10-28, `openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing` and `…-phase-2-overview-and-budget`). Phase 3 (v0.3) approved and archived (2026-10-29, -`…/2026-10-29-phase-3-snapshots-and-queue`). siding today: `setup [--port] [--budget] -[--builds]`, `teardown [--snapshots]`, `up [--wait] [--force] [--seed]`, `status [--wait]`, -`ps`, `logs`, `open`, `down [--volumes]`, `snapshot save|ls|rm`, `gc`, `lint`, `ui`, `skill`; -routing by host and path through `siding-proxy`; a memory budget with a start queue and learned -peaks; build slots; `just smoke-stacks`, `just build-darwin`. Next: Phase 4 -(`phase-4-idle-and-wake`, v0.4), starting with `/opsx:propose`. +`…/2026-10-29-phase-3-snapshots-and-queue`). Phase 4 (v0.4) approved and archived (2026-10-29, +`…/2026-10-29-phase-4-idle-and-wake`); its spike dropped Sablier (see its design.md). siding +today: `setup [--port] [--budget] [--builds] [--idle]`, `teardown [--snapshots]`, `up [--wait] +[--force] [--seed]`, `status [--wait]`, `ps`, `logs`, `open`, `down [--volumes]`, +`snapshot save|ls|rm`, `gc`, `lint`, `ui`, `skill`; routing by host and path through +`siding-proxy`; a memory budget with a start queue and learned peaks, where idle stacks make +room (the verdict `Stopped`); build slots; `just smoke-stacks`, `just build-darwin`. Next: +Phase 5 (`phase-5-remote-host`, v0.5), starting with `/opsx:propose`. ## Working agreements @@ -139,8 +141,8 @@ requests and on `main`, all inside the devcontainer image. label `dev.siding.shared`. Snapshot volumes: `dev.siding.snapshot` (name), `dev.siding.snapshot-volume` (the stack's volume key), `dev.siding.snapshot-created`, plus `dev.siding.repo` and `dev.siding.machine`. On `siding-proxy`: `dev.siding.budget` (the memory - budget in bytes), `dev.siding.builds` (build slots) and `dev.siding.host-alias` (what - `host.docker.internal` resolves to in the proxy). + budget in bytes), `dev.siding.builds` (build slots), `dev.siding.idle` (the idle threshold in seconds, `0` + for off) and `dev.siding.host-alias` (what `host.docker.internal` resolves to in the proxy). - Compose project `-`, host `..localhost`, proxy `siding-proxy` on the network `siding`, override `.siding/override.yml`. - Docker state is read through the Docker Engine API (Go SDK); lifecycle operations invoke diff --git a/README.md b/README.md index 6a8b9d1..f9e897d 100644 --- a/README.md +++ b/README.md @@ -3,9 +3,9 @@ siding gives every git worktree its own Docker Compose stack, at its own `*.localhost` address, and tells you and your coding agents whether it actually runs and what it costs. -> **Pre-alpha (v0.3).** Stacks per worktree, routing through one local proxy, the verdict of -> `siding status`, the overview `siding ui`, a memory budget with a queue and learned peaks, -> snapshots and seeding, a build limit, `gc` and `lint` work; expect +> **Pre-alpha (v0.4).** Stacks per worktree, routing through one local proxy, the verdict of +> `siding status`, the overview `siding ui`, a memory budget with a queue, learned peaks and +> idle stacks that make room, snapshots and seeding, a build limit, `gc` and `lint` work; expect > rough edges and changes before v1. There are no releases > yet: build it from source (see below). The plan is in [openspec/changes/](openspec/changes) > (one change per phase), what siding does today in [openspec/specs/](openspec/specs). @@ -52,14 +52,15 @@ ports, and puts siding's labels and routes on the services. `.siding/` ignores i ### Does it run? ```sh -siding status # Ready, Starting, Failed or TimedOut, per service -siding status --wait # until Ready or Failed, at most --timeout (3m) +siding status # Ready, Starting, Failed, TimedOut or Stopped, per service +siding status --wait # until Ready, Failed or Stopped, at most --timeout (3m) siding ps # every stack on this machine: state, address, memory ``` `Ready` means every service with a healthcheck is healthy and every service with a route answers through the proxy. A failure names the service, the reason, a hint and its last log -lines. Exit codes: 0 Ready, 1 Failed, 2 TimedOut or still starting. Every command prints one +lines. `Stopped` means no container runs and none failed; `siding up` starts it again. Exit +codes: 0 Ready, 1 Failed, 2 TimedOut, Stopped or still starting. Every command prints one JSON document with `--json`, for scripts and agents. ### Before the first up: lint @@ -143,6 +144,25 @@ began to wait. `siding ps` and `siding ui` show the line. A waiting `up` that yo Ctrl-C, or that is killed, leaves the line. `siding ps` shows the total against the budget. +Before a stack waits or is refused, idle stacks make room for it. `up` stops other stacks that +had no request through the proxy for 5 minutes, the one idle longest first, as many as the new +stack needs, and only when that makes it fit; it says on stderr which ones it stopped. Stopping +removes nothing: containers stop, volumes and networks stay. A stopped stack reports the +verdict `Stopped` (exit code 2), and `siding up` in its worktree starts it again, in Compose's +order; a request to its address does not start it (the proxy answers 404). + +```sh +siding setup --idle 15m # another idle threshold; "default" is 5m +siding setup --idle off # never stop a stack to make room +``` + +A request is anything that passes the proxy, `siding status` checks included. Work that does +not, such as tests through `docker compose exec` or a queue worker, does not count, so such a +stack can be stopped while it is busy; raise the threshold or turn it off if that matters. The +proxy records only the router and the time of each request, in its container log, which Docker +keeps at 30 MiB; no URL, header or cookie is kept. After upgrading siding, run `siding setup` +once, so the proxy records requests: until then, no stack is stopped to make room. + ### Overview ```sh diff --git a/internal/cli/cli.go b/internal/cli/cli.go index b35a334..17dbee6 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -20,7 +20,7 @@ import ( ) // Exit codes. They are part of the contract with agents, like the --json documents: 0 Ready, -// 1 Failed, 2 TimedOut (or Starting without --wait), 3 NoCapacity. A command that fails with +// 1 Failed, 2 TimedOut or Stopped (or Starting without --wait), 3 NoCapacity. A command that fails with // an Error document exits 1. const ( exitOK = 0 @@ -95,6 +95,7 @@ type Docker interface { VolumeSizes(ctx context.Context) (map[string]int64, error) RunHelper(ctx context.Context, h docker.Helper) (exit int, output []string, err error) Logs(ctx context.Context, id string, tail int) ([]string, error) + Stdout(ctx context.Context, id string) ([]string, error) Memory(ctx context.Context, id string) (uint64, error) } diff --git a/internal/cli/down.go b/internal/cli/down.go index d7b79ec..4891e74 100644 --- a/internal/cli/down.go +++ b/internal/cli/down.go @@ -3,6 +3,7 @@ package cli import ( "context" "fmt" + "io" "sort" "strings" @@ -76,6 +77,12 @@ func (a *app) down(cmd *cobra.Command, f downFlags) (downDocument, error) { return doc, nil } +// stopProject stops every container of a project by name, without files and without removing +// anything; siding up starts it again. +func (a *app) stopProject(ctx context.Context, project string, stderr io.Writer) error { + return a.opts.Compose.Run(ctx, compose.Invocation{Dir: a.opts.Home, Project: project}, []string{"stop"}, stderr, stderr) +} + // keepsWork refuses to remove data while the worktree has changes or the branch is unmerged. func (a *app) keepsWork(ctx context.Context, st existing) error { dirty, err := a.opts.Git.Dirty(ctx, st.wt.Top) diff --git a/internal/cli/down_test.go b/internal/cli/down_test.go index e00b202..7ce9720 100644 --- a/internal/cli/down_test.go +++ b/internal/cli/down_test.go @@ -158,3 +158,17 @@ func TestDownOnlyData(t *testing.T) { t.Errorf("status: exit %d, %s", code, stdout) } } + +func TestStopProject(t *testing.T) { + t.Parallel() + e := newEnv(t) + a := &app{opts: e.opts} + if err := a.stopProject(t.Context(), "shop-main", &strings.Builder{}); err != nil { + t.Fatal(err) + } + runs := e.compose.recorded() + // By project name, without a compose file, and it removes nothing. + if len(runs) != 1 || runs[0].inv.Project != "shop-main" || len(runs[0].inv.Files) != 0 || !slices.Equal(runs[0].args, []string{"stop"}) { + t.Errorf("runs %+v", runs) + } +} diff --git a/internal/cli/env_test.go b/internal/cli/env_test.go index a396a2d..c0b6bb1 100644 --- a/internal/cli/env_test.go +++ b/internal/cli/env_test.go @@ -71,7 +71,7 @@ func newEnv(t *testing.T) *env { t: t, base: base, repo: filepath.Join(base, "shop"), - docker: &fakeDocker{logs: map[string][]string{}, memory: map[string]uint64{}, memTotal: 16 << 30}, + docker: &fakeDocker{logs: map[string][]string{}, stdout: map[string][]string{}, memory: map[string]uint64{}, memTotal: 16 << 30}, compose: &fakeCompose{config: testConfig}, probe: &fakeProbe{answers: map[string]probe.Response{}}, browser: &fakeBrowser{}, @@ -114,6 +114,7 @@ func newEnv(t *testing.T) *env { var publishedPort = regexp.MustCompile(`"127\.0\.0\.1:(\d+):80"`) var budgetLabel = regexp.MustCompile(`dev\.siding\.budget: "(\d+)"`) var buildsLabel = regexp.MustCompile(`dev\.siding\.builds: "(\d+)"`) +var idleLabel = regexp.MustCompile(`dev\.siding\.idle: "(\d+)"`) var hostAliasLabel = regexp.MustCompile(`dev\.siding\.host-alias: "([^"]+)"`) // composeProxy does what docker compose up does for the proxy's project: it recreates the @@ -149,8 +150,16 @@ func (e *env) composeProxy(r composeRun) { if m := hostAliasLabel.FindStringSubmatch(string(data)); m != nil { labels[stack.LabelHostAlias] = m[1] } + if m := idleLabel.FindStringSubmatch(string(data)); m != nil { + labels[stack.LabelIdle] = m[1] + } + var cmd []string + if strings.Contains(string(data), proxy.AccessLogFlag) { + cmd = []string{proxy.AccessLogFlag} + } e.docker.proxy = &docker.Container{ID: fmt.Sprintf("proxy-%d", e.docker.proxyGen), Name: "siding-proxy", Image: proxy.Image, - State: "running", Labels: labels, Ports: []docker.Port{{HostIP: "127.0.0.1", HostPort: port, ContainerPort: 80}}} + State: "running", Labels: labels, Cmd: cmd, StartedAt: e.clock.Now(), + Ports: []docker.Port{{HostIP: "127.0.0.1", HostPort: port, ContainerPort: 80}}} } func (e *env) git(args ...string) { @@ -237,8 +246,10 @@ type fakeDocker struct { networks []docker.Network volumes []docker.Volume logs map[string][]string - memory map[string]uint64 - ensured []string + // stdout is what each container wrote to stdout, e.g. the proxy's access log. + stdout map[string][]string + memory map[string]uint64 + ensured []string } func (d *fakeDocker) add(c docker.Container) { @@ -409,6 +420,12 @@ func (d *fakeDocker) Logs(_ context.Context, id string, tail int) ([]string, err return lines, nil } +func (d *fakeDocker) Stdout(_ context.Context, id string) ([]string, error) { + d.mu.Lock() + defer d.mu.Unlock() + return slices.Clone(d.stdout[id]), nil +} + func (d *fakeDocker) Memory(_ context.Context, id string) (uint64, error) { d.mu.Lock() defer d.mu.Unlock() diff --git a/internal/cli/json.go b/internal/cli/json.go index 1ff591b..cae6e09 100644 --- a/internal/cli/json.go +++ b/internal/cli/json.go @@ -115,6 +115,8 @@ type setupDocument struct { BudgetBytes int64 `json:"budgetBytes"` // Builds is how many stacks may build images at once. Builds int `json:"builds"` + // Idle is the idle threshold, e.g. 5m0s, or off. + Idle string `json:"idle"` } // stackDocument is what up prints without --wait. @@ -136,11 +138,13 @@ type serviceAddress struct { // statusDocument is the verdict of status and up --wait. type statusDocument struct { header - // Verdict is Ready, Starting, Failed or TimedOut. + // Verdict is Ready, Starting, Failed, TimedOut or Stopped. Verdict string `json:"verdict"` Stack stackRef `json:"stack"` Services []serviceStatus `json:"services,omitempty"` Failure *failureDocument `json:"failure,omitempty"` + // Hint says what to do about a Stopped stack. + Hint string `json:"hint,omitempty"` } type serviceStatus struct { diff --git a/internal/cli/makeroom.go b/internal/cli/makeroom.go new file mode 100644 index 0000000..ac8f0ec --- /dev/null +++ b/internal/cli/makeroom.go @@ -0,0 +1,151 @@ +package cli + +import ( + "context" + "fmt" + "io" + "maps" + "slices" + "strings" + "time" + + "github.com/tweinmann/siding/internal/budget" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/idle" + "github.com/tweinmann/siding/internal/stack" +) + +// roomInterval is how often a stack waiting in line reads the proxy's log to make room; the +// log can hold 30 MiB, too much for every check of the line. +const roomInterval = 30 * time.Second + +// idleStack is a running stack nobody requested for the idle threshold. +type idleStack struct { + project string + since time.Time + memory int64 +} + +// roomMaker stops idle stacks when a new one does not fit (spec idle-stop, design D5). It says +// once per up when the proxy keeps no record of requests. +type roomMaker struct { + a *app + project string + stderr io.Writer + told bool + lastTry time.Time +} + +// make stops the idle stacks, longest idle first, that the stack needs to fit, and reports +// whether it stopped any. It stops none when even all of them would not make it fit. +func (r *roomMaker) make(ctx context.Context, check budget.Check) (bool, error) { + a := r.a + r.lastTry = a.opts.Clock.Now() + proxyContainer, err := a.opts.Docker.Container(ctx, a.opts.Proxy.Container) + if err != nil || proxyContainer == nil { + return false, err + } + threshold, on := idleThreshold(proxyContainer) + if !on { + return false, nil + } + if !idle.Recording(proxyContainer.Cmd) { + if !r.told { + warnf(r.stderr, "the proxy keeps no record of requests yet, so no idle stack can make room; siding setup turns it on") + r.told = true + } + return false, nil + } + candidates, err := r.idleStacks(ctx, proxyContainer, threshold) + if err != nil { + return false, err + } + need := check.Used + check.Estimate - check.Budget + var freed int64 + n := 0 + for n < len(candidates) && freed < need { + freed += candidates[n].memory + n++ + } + if freed < need { + return false, nil + } + now := a.opts.Clock.Now() + for _, s := range candidates[:n] { + if err := a.stopProject(ctx, s.project, r.stderr); err != nil { + return false, err + } + _, _ = fmt.Fprintf(r.stderr, "Stopped idle stack %s (no request for %s, %s) to make room for %s; siding up --wait in its worktree starts it again\n", + s.project, idleText(now.Sub(s.since)), bytesText(uint64(s.memory)), r.project) + } + return n > 0, nil +} + +// due reports whether the line may read the proxy's log again. +func (r *roomMaker) due() bool { + return r.lastTry.IsZero() || !r.a.opts.Clock.Now().Before(r.lastTry.Add(roomInterval)) +} + +// idleStacks lists the other running stacks whose last request is at least threshold ago, +// longest idle first, with their memory in use. +func (r *roomMaker) idleStacks(ctx context.Context, proxyContainer *docker.Container, threshold time.Duration) ([]idleStack, error) { + a := r.a + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelRepo: ""}) + if err != nil { + return nil, err + } + byProject := map[string][]docker.Container{} + for _, c := range cs { + if p := c.Labels[stack.LabelProject]; p != "" && p != r.project { + byProject[p] = append(byProject[p], c) + } + } + lines, err := a.opts.Docker.Stdout(ctx, proxyContainer.ID) + if err != nil { + return nil, err + } + last := idle.LastRequests(lines, slices.Collect(maps.Keys(byProject))) + now := a.opts.Clock.Now() + var out []idleStack + for project, containers := range byProject { + var started time.Time + running := false + for _, c := range containers { + running = running || c.State == "running" + if c.StartedAt.After(started) { + started = c.StartedAt + } + } + since := idle.Since(last[project], started, proxyContainer.StartedAt) + if !running || now.Sub(since) < threshold { + continue + } + memory, err := a.memory(ctx, containers) + if err != nil { + return nil, err + } + s := idleStack{project: project, since: since} + for _, m := range memory { + s.memory += int64(m) + } + out = append(out, s) + } + slices.SortFunc(out, func(x, y idleStack) int { + if c := x.since.Compare(y.since); c != 0 { + return c + } + return strings.Compare(x.project, y.project) + }) + return out, nil +} + +// idleText is a duration without a request: 45s, 20m or 2h5m. +func idleText(d time.Duration) string { + switch { + case d < time.Minute: + return fmt.Sprintf("%ds", int(d/time.Second)) + case d < time.Hour: + return fmt.Sprintf("%dm", int(d/time.Minute)) + } + return fmt.Sprintf("%dh%dm", int(d/time.Hour), int(d%time.Hour/time.Minute)) +} diff --git a/internal/cli/makeroom_test.go b/internal/cli/makeroom_test.go new file mode 100644 index 0000000..7df6dd2 --- /dev/null +++ b/internal/cli/makeroom_test.go @@ -0,0 +1,249 @@ +package cli + +import ( + "fmt" + "strings" + "testing" + "time" + + "github.com/tweinmann/siding/internal/proxy" + "github.com/tweinmann/siding/internal/stack" +) + +// roomEnv: a 4 GiB budget, a proxy that records requests and started two hours ago, and two +// stacks of the repository blog that started an hour ago: blog-main with 1 GiB, last requested +// 20 minutes ago, and blog-fix with 2.5 GiB, last requested a minute ago. This stack's +// estimate is 1 GiB (four services of 256 MiB), so it fits only without blog-main. +func roomEnv(t *testing.T) *env { + t.Helper() + const mib = 1 << 20 + e := newEnv(t) + now := e.clock.now + e.proxyUp(80) + e.docker.proxy.Labels[stack.LabelBudget] = fmt.Sprint(4096 * mib) + e.docker.proxy.Cmd = []string{"--ping=true", proxy.AccessLogFlag} + e.docker.proxy.StartedAt = now.Add(-2 * time.Hour) + for _, s := range []struct { + branch string + memory uint64 + }{{"main", 1024 * mib}, {"fix", 2560 * mib}} { + id := e.id() + id.Repo, id.Branch, id.Worktree = "blog", s.branch, "/src/blog-"+s.branch + c := container(id, "web", "running", "", 0) + c.StartedAt = now.Add(-time.Hour) + e.docker.add(c) + e.docker.memory[c.ID] = s.memory + } + e.request("blog-main-web-route", now.Add(-20*time.Minute)) + e.request("blog-fix-web-route", now.Add(-time.Minute)) + e.compose.onRun = func(r composeRun) { + switch { + case r.args[0] == "up" && r.inv.Project == "shop-main": + e.stackUp(e.id(), "api", "db", "mail", "web") + case r.args[0] == "stop": + e.docker.stopProject(r.inv.Project) + } + } + return e +} + +// request writes an access log line of the proxy, as Traefik does. +func (e *env) request(router string, at time.Time) { + e.docker.mu.Lock() + defer e.docker.mu.Unlock() + id := e.docker.proxy.ID + e.docker.stdout[id] = append(e.docker.stdout[id], + fmt.Sprintf(`{"RouterName":"%s@docker","StartUTC":"%s","level":"info","msg":"","time":"%s"}`, router, at.Format(time.RFC3339Nano), at.Format(time.RFC3339))) +} + +// stopProject does what docker compose stop does: every container of the project exits. +func (d *fakeDocker) stopProject(project string) { + d.mu.Lock() + defer d.mu.Unlock() + for i := range d.containers { + if d.containers[i].Labels[stack.LabelProject] == project { + d.containers[i].State, d.containers[i].ExitCode = "exited", 137 + } + } +} + +func (e *env) stateOf(project string) string { + e.docker.mu.Lock() + defer e.docker.mu.Unlock() + for _, c := range e.docker.containers { + if c.Labels[stack.LabelProject] == project { + return c.State + } + } + return "missing" +} + +func (e *env) stops() []string { + var out []string + for _, r := range e.compose.recorded() { + if r.args[0] == "stop" { + out = append(out, r.inv.Project) + } + } + return out +} + +func TestMakeRoom(t *testing.T) { + t.Parallel() + + t.Run("room from an idle stack", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + code, stdout, stderr := e.run("up", "--wait", "--json") + if code != exitOK || verdictOf(t, stdout) != "Ready" { + t.Fatalf("exit %d, stderr %q\n%s", code, stderr, stdout) + } + if want := "Stopped idle stack blog-main (no request for 20m, 1.0 GiB) to make room for shop-main"; !strings.Contains(stderr, want) { + t.Errorf("stderr %q lacks %q", stderr, want) + } + if strings.Contains(stderr, "Waiting for memory") { + t.Errorf("it waited although room was made: %q", stderr) + } + if got := e.stops(); len(got) != 1 || got[0] != "blog-main" || e.stateOf("blog-fix") != "running" { + t.Errorf("stopped %v, blog-fix %s", got, e.stateOf("blog-fix")) + } + }) + + t.Run("never requested", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + e.docker.stdout[e.docker.proxy.ID] = nil + e.request("blog-fix-web", e.clock.now.Add(-time.Minute)) + // blog-main started an hour ago and was never requested since. + code, _, stderr := e.run("up") + if code != exitOK || !strings.Contains(stderr, "Stopped idle stack blog-main (no request for 1h0m") { + t.Errorf("exit %d, stderr %q", code, stderr) + } + }) + + t.Run("longest idle first, only as many as needed", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + e.docker.stdout[e.docker.proxy.ID] = nil + e.request("blog-main-web", e.clock.now.Add(-20*time.Minute)) + e.request("blog-fix-web", e.clock.now.Add(-40*time.Minute)) + // Both are idle; blog-fix is idle longer, and stopping it alone is enough. + if code, _, stderr := e.run("up"); code != exitOK || len(e.stops()) != 1 || e.stops()[0] != "blog-fix" { + t.Errorf("exit %d, stopped %v, stderr %q", code, e.stops(), stderr) + } + }) + + t.Run("not enough even then", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + e.docker.proxy.Labels[stack.LabelBudget] = fmt.Sprint(3 << 30) + code, stdout, _ := e.run("up", "--json") + if code != exitNoCapacity || !strings.Contains(stdout, `"code": "NoCapacity"`) || len(e.stops()) != 0 { + t.Errorf("exit %d, stopped %v\n%s", code, e.stops(), stdout) + } + }) + + t.Run("in line", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + e.docker.stdout[e.docker.proxy.ID] = nil + e.request("blog-main-web", e.clock.now.Add(-3*time.Minute)) + e.request("blog-fix-web", e.clock.now.Add(-time.Minute)) + code, stdout, stderr := e.run("up", "--wait", "--json") + if code != exitOK || verdictOf(t, stdout) != "Ready" || !strings.Contains(stderr, "place 1 in line") || + !strings.Contains(stderr, "Stopped idle stack blog-main (no request for 5m") { + t.Fatalf("exit %d, stderr %q\n%s", code, stderr, stdout) + } + // blog-main became idle after two minutes in line; the log was read every 30 seconds. + if got := e.stops(); len(got) != 1 || got[0] != "blog-main" { + t.Errorf("stopped %v", got) + } + if waited := e.clock.now.Sub(time.Date(2026, 10, 28, 12, 0, 0, 0, time.UTC)); waited < 2*time.Minute || waited > 3*time.Minute { + t.Errorf("waited %s", waited) + } + }) + + t.Run("--force stops none", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + if code, _, stderr := e.run("up", "--force"); code != exitOK || len(e.stops()) != 0 || !strings.Contains(stderr, "--force") { + t.Errorf("exit %d, stopped %v, stderr %q", code, e.stops(), stderr) + } + }) + + t.Run("--idle off stops none", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + e.docker.proxy.Labels[stack.LabelIdle] = "0" + if code, _, _ := e.run("up"); code != exitNoCapacity || len(e.stops()) != 0 { + t.Errorf("exit %d, stopped %v", code, e.stops()) + } + }) + + t.Run("another threshold", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + e.docker.proxy.Labels[stack.LabelIdle] = "1800" // 30m: blog-main's 20 minutes are not enough + if code, _, _ := e.run("up"); code != exitNoCapacity || len(e.stops()) != 0 { + t.Errorf("exit %d, stopped %v", code, e.stops()) + } + }) + + t.Run("a proxy without the record", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + e.docker.proxy.Cmd = nil // created by siding v0.3 + code, _, stderr := e.run("up") + if code != exitNoCapacity || len(e.stops()) != 0 || !strings.Contains(stderr, "siding setup turns it on") { + t.Errorf("exit %d, stopped %v, stderr %q", code, e.stops(), stderr) + } + }) + + t.Run("the proxy was recreated", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + e.docker.stdout[e.docker.proxy.ID] = nil + e.docker.proxy.StartedAt = e.clock.now.Add(-time.Minute) + if code, _, _ := e.run("up"); code != exitNoCapacity || len(e.stops()) != 0 { + t.Errorf("exit %d, stopped %v", code, e.stops()) + } + }) + + t.Run("never its own stack", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + e.docker.stdout[e.docker.proxy.ID] = nil + e.request("blog-main-web", e.clock.now.Add(-time.Minute)) + e.request("blog-fix-web", e.clock.now.Add(-time.Minute)) + // This worktree's stack runs, idle for an hour; its next up must not stop it. + for _, s := range []string{"api", "db", "mail", "web"} { + c := container(e.id(), s, "running", "", 0) + c.StartedAt = e.clock.now.Add(-time.Hour) + e.docker.add(c) + } + if code, _, _ := e.run("up"); code != exitNoCapacity || len(e.stops()) != 0 { + t.Errorf("exit %d, stopped %v", code, e.stops()) + } + }) + + t.Run("stopped stacks use nothing", func(t *testing.T) { + t.Parallel() + e := roomEnv(t) + // blog-fix held 2.5 GiB before it was stopped. + e.docker.stopProject("blog-fix") + if code, _, stderr := e.run("up"); code != exitOK || len(e.stops()) != 0 { + t.Errorf("exit %d, stopped %v, stderr %q", code, e.stops(), stderr) + } + }) +} + +func TestIdleText(t *testing.T) { + t.Parallel() + for d, want := range map[time.Duration]string{ + 45 * time.Second: "45s", 20*time.Minute + 30*time.Second: "20m", 2*time.Hour + 5*time.Minute: "2h5m", + } { + if got := idleText(d); got != want { + t.Errorf("%s: %q, want %q", d, got, want) + } + } +} diff --git a/internal/cli/setup.go b/internal/cli/setup.go index cfb04d1..22590b5 100644 --- a/internal/cli/setup.go +++ b/internal/cli/setup.go @@ -8,6 +8,7 @@ import ( "path/filepath" "strconv" "strings" + "time" "github.com/spf13/cobra" @@ -27,8 +28,10 @@ func newSetupCommand(a *app) *cobra.Command { Short: "Start the shared proxy siding-proxy on 127.0.0.1", Long: `Start the shared proxy siding-proxy: Traefik on the network "siding", published on 127.0.0.1 only. Without --port it keeps the port of a running proxy, and uses 80 otherwise; -without --budget it keeps the budget set before. Running setup again recreates the proxy only -when something differs.`, +without --budget it keeps the budget set before, and without --idle the idle threshold. When a +new stack does not fit into the budget, up stops other stacks that had no request through the +proxy for the idle threshold (5 minutes by default). Running setup again recreates the proxy +only when something differs.`, Args: noArgs, RunE: func(cmd *cobra.Command, _ []string) error { if f.port < 0 || f.port > 65535 { @@ -42,21 +45,26 @@ when something differs.`, return printJSON(cmd.OutOrStdout(), doc) } verb := map[string]string{"created": "started", "updated": "restarted", "unchanged": "already runs"}[doc.State] - _, err = fmt.Fprintf(cmd.OutOrStdout(), "%s %s on %s, memory budget %s, %d build slots\n", doc.Container, verb, - address("*.localhost", doc.Port, "/"), bytesText(uint64(doc.BudgetBytes)), doc.Builds) + idle := "idle stacks make room after " + doc.Idle + if doc.Idle == idleOff { + idle = "idle stacks are never stopped" + } + _, err = fmt.Fprintf(cmd.OutOrStdout(), "%s %s on %s, memory budget %s, %d build slots, %s\n", doc.Container, verb, + address("*.localhost", doc.Port, "/"), bytesText(uint64(doc.BudgetBytes)), doc.Builds, idle) return err }, } cmd.Flags().IntVar(&f.port, "port", 0, "port on 127.0.0.1 for the proxy (default: the running proxy's, else 80)") cmd.Flags().StringVar(&f.builds, "builds", "", "how many stacks may build images at once; \"default\" for half of Docker's CPUs (default: the number set before)") + cmd.Flags().StringVar(&f.idle, "idle", "", "how long a stack has had no request before up may stop it to make room, e.g. 15m; \"off\" to never stop one, \"default\" for 5m (default: the threshold set before)") cmd.Flags().StringVar(&f.budget, "budget", "", "memory budget for all stacks, e.g. 12g; \"default\" for 75% of Docker's memory (default: the budget set before)") registerJSON(cmd, &asJSON) return cmd } type setupFlags struct { - port int - budget, builds string + port int + budget, builds, idle string } func (a *app) setup(cmd *cobra.Command, f setupFlags) (setupDocument, error) { @@ -104,6 +112,10 @@ func (a *app) setup(cmd *cobra.Command, f setupFlags) (setupDocument, error) { } } + if settings.Idle, err = idleSetting(f.idle, current); err != nil { + return setupDocument{}, err + } + if _, err := a.opts.Docker.EnsureNetwork(ctx, cfg.Network, map[string]string{stack.LabelMachine: machine}); err != nil { return setupDocument{}, err } @@ -151,6 +163,10 @@ func (a *app) setup(cmd *cobra.Command, f setupFlags) (setupDocument, error) { if doc.Builds, err = a.builds(ctx, after); err != nil { return setupDocument{}, err } + doc.Idle = idleOff + if d, on := idleThreshold(after); on { + doc.Idle = d.String() + } return doc, nil } @@ -168,6 +184,61 @@ func (a *app) builds(ctx context.Context, proxyContainer *docker.Container) (int return max(1, cpus/2), nil } +const ( + // defaultIdle is the idle threshold without the label dev.siding.idle. + defaultIdle = 5 * time.Minute + // minIdle is the shortest threshold setup accepts. + minIdle = 10 * time.Second + idleOff = "off" +) + +// idleSetting is the label dev.siding.idle setup writes for --idle: the current one without the +// flag, none for "default", 0 for "off", else the duration in seconds. +func idleSetting(flag string, current *docker.Container) (*int64, error) { + var seconds int64 + switch flag { + case "": + if current == nil { + return nil, nil + } + n, err := strconv.ParseInt(current.Labels[stack.LabelIdle], 10, 64) + if err != nil || n < 0 { + return nil, nil + } + seconds = n + case "default": + return nil, nil + case idleOff: + default: + d, err := time.ParseDuration(flag) + if err != nil || d < minIdle { + return nil, fail.New(fail.InvalidArguments, "--idle must be a duration of at least %s, \"off\" or \"default\", got %q", minIdle, flag) + } + seconds = int64(d / time.Second) + } + return &seconds, nil +} + +// idleThreshold is how long a stack has had no request before up may stop it to make room +// (spec idle-stop), and false when making room is off. +func idleThreshold(proxyContainer *docker.Container) (time.Duration, bool) { + if proxyContainer == nil { + return defaultIdle, true + } + v, ok := proxyContainer.Labels[stack.LabelIdle] + if !ok { + return defaultIdle, true + } + n, err := strconv.ParseInt(v, 10, 64) + switch { + case err != nil || n < 0: + return defaultIdle, true + case n == 0: + return 0, false + } + return time.Duration(n) * time.Second, true +} + // hostPort is the port on the host a proxy container publishes, or 0. func hostPort(c *docker.Container) int { if c == nil { diff --git a/internal/cli/setup_test.go b/internal/cli/setup_test.go index f94710c..606d53b 100644 --- a/internal/cli/setup_test.go +++ b/internal/cli/setup_test.go @@ -9,6 +9,8 @@ import ( "testing" "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/proxy" + "github.com/tweinmann/siding/internal/stack" "github.com/tweinmann/siding/internal/testutil" ) @@ -66,7 +68,7 @@ func TestSetup(t *testing.T) { e := newEnv(t) setupDoc(t, e) code, stdout, _ := e.run("setup") - if code != exitOK || stdout != "siding-proxy already runs on http://*.localhost/, memory budget 12.0 GiB, 4 build slots\n" { + if code != exitOK || stdout != "siding-proxy already runs on http://*.localhost/, memory budget 12.0 GiB, 4 build slots, idle stacks make room after 5m0s\n" { t.Errorf("exit %d, stdout %q", code, stdout) } }) @@ -130,6 +132,52 @@ func TestSetup(t *testing.T) { } }) + t.Run("idle threshold", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + if doc := setupDoc(t, e); doc.Idle != "5m0s" || e.docker.proxy.Labels[stack.LabelIdle] != "" { + t.Errorf("default: %+v, labels %v", doc, e.docker.proxy.Labels) + } + if doc := setupDoc(t, e, "--idle", "15m"); doc.Idle != "15m0s" || e.docker.proxy.Labels[stack.LabelIdle] != "900" { + t.Errorf("set: %+v", doc) + } + // Another setting keeps it, as the spec's scenario "Another threshold" says. + if doc := setupDoc(t, e, "--port", "8080"); doc.Idle != "15m0s" || e.docker.proxy.Labels[stack.LabelIdle] != "900" { + t.Errorf("kept: %+v", doc) + } + if doc := setupDoc(t, e, "--idle", "off"); doc.Idle != "off" || e.docker.proxy.Labels[stack.LabelIdle] != "0" { + t.Errorf("off: %+v", doc) + } + if doc := setupDoc(t, e); doc.State != "unchanged" || doc.Idle != "off" { + t.Errorf("off kept: %+v", doc) + } + if code, stdout, _ := e.run("setup"); code != exitOK || !strings.Contains(stdout, "idle stacks are never stopped") { + t.Errorf("text when off: %q", stdout) + } + if doc := setupDoc(t, e, "--idle", "default"); doc.Idle != "5m0s" { + t.Errorf("default again: %+v", doc) + } + if _, ok := e.docker.proxy.Labels[stack.LabelIdle]; ok { + t.Error("--idle default must remove the label") + } + for _, bad := range []string{"5s", "soon", "-1m"} { + before := e.docker.proxyGen + code, stdout, _ := e.run("setup", "--idle", bad, "--json") + if code != exitFailed || !strings.Contains(stdout, `"code": "InvalidArguments"`) || e.docker.proxyGen != before { + t.Errorf("--idle %s: exit %d, %s", bad, code, stdout) + } + } + }) + + t.Run("records requests", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + setupDoc(t, e) + if !slices.Contains(e.docker.proxy.Cmd, proxy.AccessLogFlag) { + t.Errorf("the proxy does not record requests: %v", e.docker.proxy.Cmd) + } + }) + t.Run("a proxy of siding v0.1", func(t *testing.T) { t.Parallel() e := newEnv(t) diff --git a/internal/cli/skill/SKILL.md b/internal/cli/skill/SKILL.md index c6f6924..27195c0 100644 --- a/internal/cli/skill/SKILL.md +++ b/internal/cli/skill/SKILL.md @@ -31,6 +31,9 @@ Read its `verdict` and the exit code: listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady` means it is still starting. Read the hint and the logs; if it is only slow, run `siding status --wait --json --timeout 5m`. +- **2, `Stopped`**: every container of the stack is stopped, nothing failed; often `up` in + another worktree stopped it to make room in the memory budget. No data is lost. Run + `siding up --wait --json` to start it again. - **3, `NoCapacity`**: the stack does not fit into the machine's memory budget; nothing was started. Stop and tell the user, with the `message` of the `Error` document. Freeing memory or raising the budget is the user's decision. @@ -40,13 +43,16 @@ An `Error` document (exit 1, `kind: Error`) means siding could not act at all; i ## More detail -- `siding status --json`: the current verdict, without waiting (exit 2 while starting). +- `siding status --json`: the current verdict and the stack's address in `stack.url`, without + waiting and without starting anything (exit 2 while starting or stopped). - `siding logs --json` or `siding logs --json --tail 200`: the last log lines per service. Values from `.env` and the services' environment appear as `***`. - `siding ps --json`: every stack on this machine, with state and memory. ## Rules +- Only the addresses siding reports work: `stack.url` and the services' `url`. Never give + `localhost:` from the compose file; siding publishes no ports. - Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file or the service labels `dev.siding.route` and `dev.siding.port` instead. - Leave stopping and removing stacks to the user; `siding down` only removes containers, and diff --git a/internal/cli/skill_test.go b/internal/cli/skill_test.go index f938dd5..9634d9c 100644 --- a/internal/cli/skill_test.go +++ b/internal/cli/skill_test.go @@ -21,13 +21,15 @@ func TestSkill(t *testing.T) { if !strings.HasPrefix(text, "---\nname: siding\ndescription: ") { t.Errorf("not a skill named siding:\n%s", text[:min(len(text), 80)]) } - for _, want := range []string{"siding up --wait --json", "siding status --json", "siding logs --json", "**0, `Ready`**", "**1, `Failed`**", "**2, `TimedOut`**", "**3, `NoCapacity`**"} { + for _, want := range []string{"siding up --wait --json", "siding status --json", "siding logs --json", "**0, `Ready`**", "**1, `Failed`**", "**2, `TimedOut`**", "**2, `Stopped`**", "**3, `NoCapacity`**", + "`stack.url`", "Never give\n `localhost:`"} { if !strings.Contains(text, want) { t.Errorf("the skill does not mention %q", want) } } // Only a human removes data, forces anything, or exposes a stack. - for _, never := range []string{"--force", "--volumes", "share", "gc"} { + // The idle threshold is the user's decision too. + for _, never := range []string{"--force", "--volumes", "--idle", "share", "gc"} { if strings.Contains(text, never) { t.Errorf("the skill mentions %q", never) } diff --git a/internal/cli/status.go b/internal/cli/status.go index 1795339..0bf55c9 100644 --- a/internal/cli/status.go +++ b/internal/cli/status.go @@ -31,7 +31,7 @@ type waitFlags struct { } func (w *waitFlags) register(cmd *cobra.Command) { - cmd.Flags().BoolVar(&w.wait, "wait", false, "wait until the stack is Ready or Failed, at most --timeout") + cmd.Flags().BoolVar(&w.wait, "wait", false, "wait until the stack is Ready, Failed or Stopped, at most --timeout") cmd.Flags().DurationVar(&w.timeout, "timeout", defaultTimeout, "how long --wait waits") } @@ -41,12 +41,13 @@ func newStatusCommand(a *app) *cobra.Command { var w waitFlags cmd := &cobra.Command{ Use: "status", - Short: "Tell whether the worktree's stack runs: Ready, Starting, Failed or TimedOut", + Short: "Tell whether the worktree's stack runs: Ready, Starting, Failed, TimedOut or Stopped", Long: `Tell whether the worktree's stack runs. Ready means every service has a container, every healthcheck is healthy, and every service with a route path answers through the proxy. -A failure names the service, the reason, a hint and its last log lines. +A failure names the service, the reason, a hint and its last log lines. Stopped means no +container runs and none failed, e.g. after up in another worktree stopped it to make room. -Exit codes: 0 Ready, 1 Failed, 2 TimedOut (or Starting without --wait).`, +Exit codes: 0 Ready, 1 Failed, 2 TimedOut, Stopped (or Starting without --wait).`, Args: noArgs, RunE: func(cmd *cobra.Command, _ []string) error { ctx := cmd.Context() @@ -89,6 +90,10 @@ func (a *app) status(ctx context.Context, st existing, p *compose.Project, port routed[r.Service] = verdict.Route{URL: address(st.id.Host(), port, r.Path), Port: r.Port} } } + healthchecks := map[string]bool{} + for name, svc := range p.Services { + healthchecks[name] = svc.Healthcheck + } start := a.opts.Clock.Now() var base map[string]int for { @@ -102,13 +107,18 @@ func (a *app) status(ctx context.Context, st existing, p *compose.Project, port base[c.ID] = c.RestartCount } } - answering := a.probe(ctx, st, routes, cs, port) final := w.wait && a.opts.Clock.Now().Sub(start) >= w.timeout - res := verdict.Evaluate(verdict.Input{ - Services: p.ServiceNames(), Containers: cs, Routes: routed, Answering: answering, + in := verdict.Input{ + Services: p.ServiceNames(), Containers: cs, Routes: routed, Healthchecks: healthchecks, RestartBase: base, Final: final, Timeout: w.timeout, Seed: seedFailure, - }) - if !w.wait || final || res.Verdict == verdict.Ready || res.Verdict == verdict.Failed { + } + // A stopped stack is not probed: nothing would answer (spec stack-status). + res := verdict.Evaluate(in) + if res.Verdict != verdict.Stopped { + in.Answering = a.probe(ctx, st, routes, cs, port) + res = verdict.Evaluate(in) + } + if !w.wait || final || res.Verdict == verdict.Ready || res.Verdict == verdict.Failed || res.Verdict == verdict.Stopped { if res.Verdict == verdict.Ready { _, _ = a.memory(ctx, cs) // learns this stack's peaks (design D7) } @@ -142,8 +152,14 @@ func (a *app) probe(ctx context.Context, st existing, routes []stack.Route, cs [ func (a *app) statusDocument(ctx context.Context, st existing, p *compose.Project, routes []stack.Route, port int, cs []docker.Container, res verdict.Result) (statusDocument, error) { doc := statusDocument{header: newHeader(kindStatus), Verdict: res.Verdict, Stack: st.ref(port)} + if res.Verdict == verdict.Stopped { + doc.Hint = stoppedHint + } for _, s := range res.Services { ss := serviceStatus{Name: s.Name, State: s.State, Health: s.Health, Answering: s.Answering, Ready: s.Ready} + if res.Verdict == verdict.Stopped { + ss.Answering = nil // not asked: a stopped stack is not probed + } if r, ok := stack.Find(routes, s.Name); ok { ss.Route = r.Path ss.URL = address(st.id.ServiceHost(s.Name), port, "/") @@ -169,6 +185,9 @@ func (a *app) statusDocument(ctx context.Context, st existing, p *compose.Projec return doc, nil } +// stoppedHint is what a Stopped verdict tells: siding stopped it to make room, or someone did. +const stoppedHint = "every container of the stack is stopped, perhaps to make room for another stack; siding up --wait starts it again" + // failing picks the container of a service whose logs explain its failure: one that is not // running, else the first. func failing(cs []docker.Container) docker.Container { @@ -222,6 +241,9 @@ func printStatusText(w io.Writer, doc statusDocument) error { rows = append(rows, []string{s.Name, state, reach, where}) } b.WriteString(table(" ", rows)) + if doc.Hint != "" { + fmt.Fprintf(&b, "\n%s\n", doc.Hint) + } if f := doc.Failure; f != nil { fmt.Fprintf(&b, "\n%s: %s, %s\n", f.Service, f.Reason, f.Hint) if len(f.Logs) > 0 { diff --git a/internal/cli/status_test.go b/internal/cli/status_test.go index f6a357e..f23085d 100644 --- a/internal/cli/status_test.go +++ b/internal/cli/status_test.go @@ -31,6 +31,27 @@ func TestStatus(t *testing.T) { } }) + t.Run("stopped", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + for _, svc := range []string{"api", "db", "mail", "web"} { + e.docker.add(container(e.id(), svc, "exited", "", 137)) + } + // --wait ends at once: a stopped stack does not start by itself. + code, stdout, stderr := e.run("status", "--wait", "--json") + if code != exitTimedOut || stderr != "" { + t.Fatalf("exit %d, stderr %q, stdout %s", code, stderr, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "status-stopped.json"), []byte(stdout)) + if len(e.probe.asked) != 0 || e.clock.slept != 0 { + t.Errorf("probes %v, slept %d", e.probe.asked, e.clock.slept) + } + if code, out, _ := e.run("status"); code != exitTimedOut || !strings.Contains(out, "shop-main: Stopped") || !strings.Contains(out, "siding up --wait starts it again") { + t.Errorf("text: exit %d\n%s", code, out) + } + }) + t.Run("failed with the log tail, redacted", func(t *testing.T) { t.Parallel() e := newEnv(t) diff --git a/internal/cli/testdata/json/setup.json b/internal/cli/testdata/json/setup.json index 503c434..4220cef 100644 --- a/internal/cli/testdata/json/setup.json +++ b/internal/cli/testdata/json/setup.json @@ -8,5 +8,6 @@ "url": "http://*.localhost/", "state": "created", "budgetBytes": 12884901888, - "builds": 4 + "builds": 4, + "idle": "5m0s" } diff --git a/internal/cli/testdata/json/skill.json b/internal/cli/testdata/json/skill.json index ed127e5..a5788d1 100644 --- a/internal/cli/testdata/json/skill.json +++ b/internal/cli/testdata/json/skill.json @@ -2,5 +2,5 @@ "apiVersion": "siding.dev/v1alpha1", "kind": "Skill", "name": "siding", - "text": "---\nname: siding\ndescription: Start this worktree's Docker Compose stack with siding and wait for the verdict, to check that a change to the app actually runs. Use after changing code, Dockerfiles, compose files or configuration that the running app depends on, and before reporting such work as done.\n---\n\n# Check that the app runs with siding\n\nsiding runs this git worktree's compose file as its own stack, reachable through a local proxy\nat `http://..localhost`, and tells you whether it actually runs. Every command\nbelow prints one JSON document with `--json`.\n\n## After a change that affects the running app\n\nRun:\n\n```sh\nsiding up --wait --json\n```\n\nIt starts or updates the stack and waits, three minutes at most, for one `Status` document.\nRead its `verdict` and the exit code:\n\n- **0, `Ready`**: every service runs, every healthcheck is healthy and every routed service\n answers through the proxy. Report the address in `stack.url` (and a service's `url` if it\n matters) together with your result.\n- **1, `Failed`**: `failure` names the `service`, the `reason` (`Exited`, `Unhealthy`,\n `Restarting`, `Missing`), a `hint` and the last log lines of that service in `logs`. Fix the\n cause in that service, then run `siding up --wait --json` again.\n- **2, `TimedOut`**: the stack was not ready in time. `failure` names the first service that\n was not ready: `NotAnswering` means it runs but does not answer through the proxy (often it\n listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady`\n means it is still starting. Read the hint and the logs; if it is only slow, run\n `siding status --wait --json --timeout 5m`.\n- **3, `NoCapacity`**: the stack does not fit into the machine's memory budget; nothing was\n started. Stop and tell the user, with the `message` of the `Error` document. Freeing memory\n or raising the budget is the user's decision.\n\nAn `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`,\n`message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`.\n\n## More detail\n\n- `siding status --json`: the current verdict, without waiting (exit 2 while starting).\n- `siding logs --json` or `siding logs --json --tail 200`: the last log lines per\n service. Values from `.env` and the services' environment appear as `***`.\n- `siding ps --json`: every stack on this machine, with state and memory.\n\n## Rules\n\n- Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file\n or the service labels `dev.siding.route` and `dev.siding.port` instead.\n- Leave stopping and removing stacks to the user; `siding down` only removes containers, and\n removing data is the user's decision.\n- Do not paste log lines into places outside this conversation.\n" + "text": "---\nname: siding\ndescription: Start this worktree's Docker Compose stack with siding and wait for the verdict, to check that a change to the app actually runs. Use after changing code, Dockerfiles, compose files or configuration that the running app depends on, and before reporting such work as done.\n---\n\n# Check that the app runs with siding\n\nsiding runs this git worktree's compose file as its own stack, reachable through a local proxy\nat `http://..localhost`, and tells you whether it actually runs. Every command\nbelow prints one JSON document with `--json`.\n\n## After a change that affects the running app\n\nRun:\n\n```sh\nsiding up --wait --json\n```\n\nIt starts or updates the stack and waits, three minutes at most, for one `Status` document.\nRead its `verdict` and the exit code:\n\n- **0, `Ready`**: every service runs, every healthcheck is healthy and every routed service\n answers through the proxy. Report the address in `stack.url` (and a service's `url` if it\n matters) together with your result.\n- **1, `Failed`**: `failure` names the `service`, the `reason` (`Exited`, `Unhealthy`,\n `Restarting`, `Missing`), a `hint` and the last log lines of that service in `logs`. Fix the\n cause in that service, then run `siding up --wait --json` again.\n- **2, `TimedOut`**: the stack was not ready in time. `failure` names the first service that\n was not ready: `NotAnswering` means it runs but does not answer through the proxy (often it\n listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady`\n means it is still starting. Read the hint and the logs; if it is only slow, run\n `siding status --wait --json --timeout 5m`.\n- **2, `Stopped`**: every container of the stack is stopped, nothing failed; often `up` in\n another worktree stopped it to make room in the memory budget. No data is lost. Run\n `siding up --wait --json` to start it again.\n- **3, `NoCapacity`**: the stack does not fit into the machine's memory budget; nothing was\n started. Stop and tell the user, with the `message` of the `Error` document. Freeing memory\n or raising the budget is the user's decision.\n\nAn `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`,\n`message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`.\n\n## More detail\n\n- `siding status --json`: the current verdict and the stack's address in `stack.url`, without\n waiting and without starting anything (exit 2 while starting or stopped).\n- `siding logs --json` or `siding logs --json --tail 200`: the last log lines per\n service. Values from `.env` and the services' environment appear as `***`.\n- `siding ps --json`: every stack on this machine, with state and memory.\n\n## Rules\n\n- Only the addresses siding reports work: `stack.url` and the services' `url`. Never give\n `localhost:` from the compose file; siding publishes no ports.\n- Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file\n or the service labels `dev.siding.route` and `dev.siding.port` instead.\n- Leave stopping and removing stacks to the user; `siding down` only removes containers, and\n removing data is the user's decision.\n- Do not paste log lines into places outside this conversation.\n" } diff --git a/internal/cli/testdata/json/status-stopped.json b/internal/cli/testdata/json/status-stopped.json new file mode 100644 index 0000000..2060c62 --- /dev/null +++ b/internal/cli/testdata/json/status-stopped.json @@ -0,0 +1,40 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Status", + "verdict": "Stopped", + "stack": { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "url": "http://shop.main.localhost/" + }, + "services": [ + { + "name": "api", + "state": "exited", + "route": "/api", + "url": "http://api.shop.main.localhost/", + "ready": false + }, + { + "name": "db", + "state": "exited", + "ready": false + }, + { + "name": "mail", + "state": "exited", + "url": "http://mail.shop.main.localhost/", + "ready": false + }, + { + "name": "web", + "state": "exited", + "route": "/", + "url": "http://web.shop.main.localhost/", + "ready": false + } + ], + "hint": "every container of the stack is stopped, perhaps to make room for another stack; siding up --wait starts it again" +} diff --git a/internal/cli/testdata/skill/SKILL.md b/internal/cli/testdata/skill/SKILL.md index c6f6924..27195c0 100644 --- a/internal/cli/testdata/skill/SKILL.md +++ b/internal/cli/testdata/skill/SKILL.md @@ -31,6 +31,9 @@ Read its `verdict` and the exit code: listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady` means it is still starting. Read the hint and the logs; if it is only slow, run `siding status --wait --json --timeout 5m`. +- **2, `Stopped`**: every container of the stack is stopped, nothing failed; often `up` in + another worktree stopped it to make room in the memory budget. No data is lost. Run + `siding up --wait --json` to start it again. - **3, `NoCapacity`**: the stack does not fit into the machine's memory budget; nothing was started. Stop and tell the user, with the `message` of the `Error` document. Freeing memory or raising the budget is the user's decision. @@ -40,13 +43,16 @@ An `Error` document (exit 1, `kind: Error`) means siding could not act at all; i ## More detail -- `siding status --json`: the current verdict, without waiting (exit 2 while starting). +- `siding status --json`: the current verdict and the stack's address in `stack.url`, without + waiting and without starting anything (exit 2 while starting or stopped). - `siding logs --json` or `siding logs --json --tail 200`: the last log lines per service. Values from `.env` and the services' environment appear as `***`. - `siding ps --json`: every stack on this machine, with state and memory. ## Rules +- Only the addresses siding reports work: `stack.url` and the services' `url`. Never give + `localhost:` from the compose file; siding publishes no ports. - Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file or the service labels `dev.siding.route` and `dev.siding.port` instead. - Leave stopping and removing stacks to the user; `siding down` only removes containers, and diff --git a/internal/cli/up.go b/internal/cli/up.go index 8fbd795..436d5a6 100644 --- a/internal/cli/up.go +++ b/internal/cli/up.go @@ -113,21 +113,35 @@ func (a *app) checkBudget(ctx context.Context, project, repo string, p *compose. if err != nil || check.Fits() { return err } - switch { - case force: + if force { warnf(stderr, "%s; starting it anyway (--force)", capacityMessage(project, check)) return nil - case w.wait: - return a.waitInLine(ctx, project, repo, p, check, w.timeout, stderr) + } + // Idle stacks make room before the stack waits or is refused (spec idle-stop). + room := &roomMaker{a: a, project: project, stderr: stderr} + if check, err = a.fitAfterRoom(ctx, room, project, repo, p, check); err != nil || check.Fits() { + return err + } + if w.wait { + return a.waitInLine(ctx, room, project, repo, p, check, w.timeout, stderr) } return fail.New(fail.NoCapacity, "%s", capacityMessage(project, check)).WithHint(capacityHint) } +// fitAfterRoom stops idle stacks when that makes the stack fit, and checks the budget again. +func (a *app) fitAfterRoom(ctx context.Context, room *roomMaker, project, repo string, p *compose.Project, check budget.Check) (budget.Check, error) { + stopped, err := room.make(ctx, check) + if err != nil || !stopped { + return check, err + } + return a.fit(ctx, project, repo, p) +} + // queueInterval is how often a waiting up looks at the line and the budget again. const queueInterval = 2 * time.Second // waitInLine waits until this up is first in line and its stack fits, at most timeout. -func (a *app) waitInLine(ctx context.Context, project, repo string, p *compose.Project, check budget.Check, timeout time.Duration, stderr io.Writer) error { +func (a *app) waitInLine(ctx context.Context, room *roomMaker, project, repo string, p *compose.Project, check budget.Check, timeout time.Duration, stderr io.Writer) error { line := queue.Open(a.opts.Home, a.opts.Alive) start := a.opts.Clock.Now() leave, err := line.Enter(queue.Entry{Project: project, PID: a.opts.PID, Estimate: check.Estimate, Since: start}) @@ -150,6 +164,11 @@ func (a *app) waitInLine(ctx context.Context, project, repo string, p *compose.P if check, err = a.fit(ctx, project, repo, p); err != nil { return err } + if !check.Fits() && room.due() { + if check, err = a.fitAfterRoom(ctx, room, project, repo, p, check); err != nil { + return err + } + } if check.Fits() { return nil } diff --git a/internal/docker/docker.go b/internal/docker/docker.go index 02c276e..7224ee2 100644 --- a/internal/docker/docker.go +++ b/internal/docker/docker.go @@ -39,6 +39,10 @@ type Container struct { RestartCount int OOMKilled bool Ports []Port + // StartedAt is when the container last started; zero when it never did. + StartedAt time.Time + // Cmd is the container's command, its arguments after the image's entrypoint. + Cmd []string } // Port is a published port. @@ -306,6 +310,7 @@ func fromInspect(in container.InspectResponse) Container { if in.Config != nil { ct.Image = in.Config.Image ct.Labels = in.Config.Labels + ct.Cmd = in.Config.Cmd } if ct.Labels == nil { ct.Labels = map[string]string{} @@ -314,6 +319,9 @@ func fromInspect(in container.InspectResponse) Container { ct.State = string(s.Status) ct.ExitCode = s.ExitCode ct.OOMKilled = s.OOMKilled + if t, err := time.Parse(time.RFC3339Nano, s.StartedAt); err == nil && t.Year() > 1 { + ct.StartedAt = t + } if h := s.Health; h != nil { ct.Health = string(h.Status) if n := len(h.Log); n > 0 && h.Log[n-1] != nil { @@ -442,11 +450,20 @@ func (c *Client) Volumes(ctx context.Context, labels map[string]string) ([]Volum // Logs returns the last tail lines a container wrote to stdout and stderr, in order. func (c *Client) Logs(ctx context.Context, id string, tail int) ([]string, error) { + return c.logs(ctx, id, client.ContainerLogsOptions{ShowStdout: true, ShowStderr: true, Tail: strconv.Itoa(tail)}) +} + +// Stdout returns every line a container wrote to stdout that Docker still keeps, in order. +func (c *Client) Stdout(ctx context.Context, id string) ([]string, error) { + return c.logs(ctx, id, client.ContainerLogsOptions{ShowStdout: true, Tail: "all"}) +} + +func (c *Client) logs(ctx context.Context, id string, opts client.ContainerLogsOptions) ([]string, error) { api, err := c.connect() if err != nil { return nil, err } - rc, err := api.ContainerLogs(ctx, id, client.ContainerLogsOptions{ShowStdout: true, ShowStderr: true, Tail: strconv.Itoa(tail)}) + rc, err := api.ContainerLogs(ctx, id, opts) if err != nil { return nil, fmt.Errorf("logs of %s: %w", id, err) } diff --git a/internal/docker/docker_test.go b/internal/docker/docker_test.go index 1c87556..9c97270 100644 --- a/internal/docker/docker_test.go +++ b/internal/docker/docker_test.go @@ -4,6 +4,7 @@ import ( "net/netip" "reflect" "testing" + "time" "github.com/moby/moby/api/types/container" "github.com/moby/moby/api/types/network" @@ -19,10 +20,12 @@ func TestFromInspect(t *testing.T) { ID: "abc", Name: "/siding-proxy", RestartCount: 2, - Config: &container.Config{Image: "traefik:v3.7.16", Labels: map[string]string{"dev.siding.machine": "m"}}, + Config: &container.Config{Image: "traefik:v3.7.16", Labels: map[string]string{"dev.siding.machine": "m"}, + Cmd: []string{"--accesslog=true"}}, State: &container.State{ - Status: container.StateRunning, - ExitCode: 0, + Status: container.StateRunning, + ExitCode: 0, + StartedAt: "2026-10-29T18:28:38.385704594Z", Health: &container.Health{Status: container.Unhealthy, Log: []*container.HealthcheckResult{ {Output: "old"}, {Output: "connection refused\n"}, }}, @@ -34,7 +37,8 @@ func TestFromInspect(t *testing.T) { want := Container{ ID: "abc", Name: "siding-proxy", Image: "traefik:v3.7.16", Labels: map[string]string{"dev.siding.machine": "m"}, State: "running", Health: "unhealthy", HealthOutput: "connection refused", RestartCount: 2, - Ports: []Port{{HostIP: "127.0.0.1", HostPort: 8080, ContainerPort: 80}}, + Ports: []Port{{HostIP: "127.0.0.1", HostPort: 8080, ContainerPort: 80}}, + StartedAt: time.Date(2026, 10, 29, 18, 28, 38, 385704594, time.UTC), Cmd: []string{"--accesslog=true"}, } if got := fromInspect(in); !reflect.DeepEqual(got, want) { t.Errorf("got %+v\nwant %+v", got, want) @@ -42,6 +46,11 @@ func TestFromInspect(t *testing.T) { if got := fromInspect(container.InspectResponse{}); got.Labels == nil { t.Error("labels must never be nil") } + // A container that never started reports Docker's zero time. + never := container.InspectResponse{State: &container.State{Status: container.StateCreated, StartedAt: "0001-01-01T00:00:00Z"}} + if got := fromInspect(never); !got.StartedAt.IsZero() { + t.Errorf("never started: %v", got.StartedAt) + } } func TestMemoryUsage(t *testing.T) { diff --git a/internal/e2e/phase4_test.go b/internal/e2e/phase4_test.go new file mode 100644 index 0000000..aa6cb24 --- /dev/null +++ b/internal/e2e/phase4_test.go @@ -0,0 +1,92 @@ +//go:build docker + +package e2e + +import ( + "context" + "strings" + "testing" + "time" +) + +// roomCompose needs 200 MiB by its limit, so with a budget of 200 MiB a second one fits only +// when the first, which uses some memory, is stopped; its volume keeps a file. +const roomCompose = `services: + web: + image: busybox:1.37.0 + command: ["sh", "-c", "mkdir -p /www && echo web > /www/index.html && exec httpd -f -p 80 -h /www"] + expose: ["80"] + labels: {dev.siding.route: "/"} + mem_limit: 200m + volumes: [data:/data] +volumes: + data: {} +` + +func TestMakingRoom(t *testing.T) { + m := newMachine(t) + // A budget for one stack of 200 MiB; a stack idle for 10 seconds makes room. + if code, out := m.siding(t.TempDir(), "setup", "--idle", "10s", "--budget", "200m", "--json"); code != 0 || !strings.Contains(out, `"idle": "10s"`) { + t.Fatalf("setup: exit %d\n%s", code, out) + } + + a := repo(t, "rooma", roomCompose, "", "feature/r") + const projectA, hostA = "siding-test-rooma-feature-r", "siding-test-rooma.feature-r.localhost" + if code, out := m.siding(a, "up", "--repo", "siding-test-rooma", "--wait", "--json"); code != 0 { + t.Fatalf("up a: exit %d\n%s", code, out) + } + composeExec(t, projectA, "web", "sh", "-c", "echo kept > /data/hello") + + t.Run("the proxy records only router and time", func(t *testing.T) { + if code, _, _ := get(t, hostA, "/account?token=not-a-real-token"); code != 404 { + t.Errorf("GET: %d, want the app's 404", code) + } + var logs string + for i := 0; i < 20 && !strings.Contains(logs, projectA); i++ { + time.Sleep(250 * time.Millisecond) + logs = run(t, "", "docker", "logs", testProxy.Container) + } + if !strings.Contains(logs, `"RouterName":"`+projectA+`-web-route@docker"`) { + t.Errorf("no line for the request:\n%s", logs) + } + if strings.Contains(logs, "not-a-real-token") || strings.Contains(logs, "/account") { + t.Errorf("the record holds the URL:\n%s", logs) + } + }) + + // No request for longer than the threshold. Not even siding status: its probes are requests. + time.Sleep(15 * time.Second) + + b := repo(t, "roomb", roomCompose, "", "feature/r") + t.Run("up stops the idle stack to make room", func(t *testing.T) { + code, out, stderr := m.sidingFull(context.Background(), b, "up", "--repo", "siding-test-roomb", "--wait", "--json") + if code != 0 || decode(t, out).Verdict != "Ready" { + t.Fatalf("up b: exit %d\n%s", code, out) + } + if !strings.Contains(stderr, "Stopped idle stack "+projectA) { + t.Errorf("stderr does not name the stopped stack:\n%s", stderr) + } + }) + + t.Run("the stopped stack", func(t *testing.T) { + code, out := m.siding(a, "status", "--json") + if code != 2 || decode(t, out).Verdict != "Stopped" || !strings.Contains(out, `"url": "http://`+hostA+`:18080/"`) { + t.Errorf("status: exit %d\n%s", code, out) + } + if code, _, _ := get(t, hostA, "/"); code != 404 { + t.Errorf("a request to the stopped stack: %d, want 404", code) + } + }) + + t.Run("up starts it again with its data", func(t *testing.T) { + if code, out := m.siding(b, "down", "--json"); code != 0 { + t.Fatalf("down b: exit %d\n%s", code, out) + } + if code, out := m.siding(a, "up", "--repo", "siding-test-rooma", "--wait", "--json"); code != 0 || decode(t, out).Verdict != "Ready" { + t.Fatalf("up a again: exit %d\n%s", code, out) + } + if got := composeExec(t, projectA, "web", "cat", "/data/hello"); got != "kept" { + t.Errorf("data %q, want kept", got) + } + }) +} diff --git a/internal/idle/idle.go b/internal/idle/idle.go new file mode 100644 index 0000000..8ce8846 --- /dev/null +++ b/internal/idle/idle.go @@ -0,0 +1,66 @@ +// Package idle tells when a stack was last requested through the proxy, from the proxy's access +// log: one JSON line per request with only the router and the time (phase 4 design D3, D4). +package idle + +import ( + "encoding/json" + "slices" + "strings" + "time" + + "github.com/tweinmann/siding/internal/proxy" +) + +// Recording reports whether a proxy with this command records its requests; a proxy of siding +// v0.3 does not. +func Recording(cmd []string) bool { + return slices.Contains(cmd, proxy.AccessLogFlag) +} + +// line is what Traefik's JSON access log keeps of a request. +type line struct { + RouterName string + StartUTC time.Time +} + +// LastRequests returns the latest request per project in the proxy's log lines. A router belongs +// to the longest project name it starts with, followed by "-": router names are +// -, so shop-main-x-web belongs to shop-main-x, not to shop-main. Lines that +// are not access log lines, such as Traefik's own, are skipped. +func LastRequests(lines []string, projects []string) map[string]time.Time { + byLength := slices.Clone(projects) + slices.SortFunc(byLength, func(a, b string) int { return len(b) - len(a) }) + last := map[string]time.Time{} + for _, l := range lines { + if !strings.HasPrefix(l, "{") { + continue + } + var req line + if err := json.Unmarshal([]byte(l), &req); err != nil || req.RouterName == "" || req.StartUTC.IsZero() { + continue + } + router, _, _ := strings.Cut(req.RouterName, "@") + for _, p := range byLength { + if strings.HasPrefix(router, p+"-") { + if req.StartUTC.After(last[p]) { + last[p] = req.StartUTC + } + break + } + } + } + return last +} + +// Since is the time a stack counts as last requested: its latest request, or the later start of +// the stack or the proxy when that is later, because the proxy's record begins when it starts +// and a new stack has not had its chance yet (spec idle-stop). +func Since(request, stackStart, proxyStart time.Time) time.Time { + latest := request + for _, t := range []time.Time{stackStart, proxyStart} { + if t.After(latest) { + latest = t + } + } + return latest +} diff --git a/internal/idle/idle_test.go b/internal/idle/idle_test.go new file mode 100644 index 0000000..ac8dedd --- /dev/null +++ b/internal/idle/idle_test.go @@ -0,0 +1,74 @@ +package idle + +import ( + "testing" + "time" +) + +func TestRecording(t *testing.T) { + t.Parallel() + if !Recording([]string{"--ping=true", "--accesslog=true"}) || Recording([]string{"--ping=true"}) { + t.Error("only a proxy with --accesslog=true records its requests") + } +} + +func TestLastRequests(t *testing.T) { + t.Parallel() + at := func(s string) time.Time { + t.Helper() + v, err := time.Parse(time.RFC3339Nano, s) + if err != nil { + t.Fatal(err) + } + return v + } + lines := []string{ + // Traefik's own log is not JSON. + `2026-10-29T18:27:13Z INF Loading plugins...`, + `{"RouterName":"shop-main-web@docker","StartUTC":"2026-10-29T18:28:38.385704594Z","level":"info","msg":"","time":"2026-10-29T18:28:38Z"}`, + `{"RouterName":"shop-main-web-route@docker","StartUTC":"2026-10-29T18:30:00Z","level":"info"}`, + // An older request after a newer one: the latest counts. + `{"RouterName":"shop-main-api@docker","StartUTC":"2026-10-29T18:29:00Z"}`, + // shop-main-x is a project of its own, not a service x of shop-main. + `{"RouterName":"shop-main-x-web@docker","StartUTC":"2026-10-29T18:31:00Z"}`, + // The overview and unknown projects belong to no stack. + `{"RouterName":"siding-ui@file","StartUTC":"2026-10-29T18:32:00Z"}`, + `{"RouterName":"other-main-web@docker","StartUTC":"2026-10-29T18:33:00Z"}`, + // Broken lines and lines without a router or a time. + `{"RouterName":"shop-fix-web@docker","StartUTC":`, + `{"StartUTC":"2026-10-29T18:34:00Z"}`, + `{"RouterName":"shop-fix-web@docker"}`, + } + got := LastRequests(lines, []string{"shop-main", "shop-main-x", "shop-fix"}) + want := map[string]time.Time{"shop-main": at("2026-10-29T18:30:00Z"), "shop-main-x": at("2026-10-29T18:31:00Z")} + if len(got) != len(want) { + t.Fatalf("got %v, want %v", got, want) + } + for p, w := range want { + if !got[p].Equal(w) { + t.Errorf("%s: %v, want %v", p, got[p], w) + } + } + if len(LastRequests(nil, []string{"shop-main"})) != 0 { + t.Error("an empty record knows no request") + } +} + +func TestSince(t *testing.T) { + t.Parallel() + base := time.Date(2026, 10, 29, 12, 0, 0, 0, time.UTC) + for _, tt := range []struct { + name string + request, stack, proxyT time.Time + want time.Time + }{ + {name: "requested after both started", request: base.Add(-2 * time.Minute), stack: base.Add(-time.Hour), proxyT: base.Add(-2 * time.Hour), want: base.Add(-2 * time.Minute)}, + {name: "never requested", stack: base.Add(-20 * time.Minute), proxyT: base.Add(-time.Hour), want: base.Add(-20 * time.Minute)}, + {name: "the proxy was recreated", stack: base.Add(-time.Hour), proxyT: base.Add(-time.Minute), want: base.Add(-time.Minute)}, + {name: "started again after its last request", request: base.Add(-time.Hour), stack: base.Add(-3 * time.Minute), proxyT: base.Add(-2 * time.Hour), want: base.Add(-3 * time.Minute)}, + } { + if got := Since(tt.request, tt.stack, tt.proxyT); !got.Equal(tt.want) { + t.Errorf("%s: %v, want %v", tt.name, got, tt.want) + } + } +} diff --git a/internal/proxy/proxy.go b/internal/proxy/proxy.go index f17c55e..9bb97aa 100644 --- a/internal/proxy/proxy.go +++ b/internal/proxy/proxy.go @@ -26,6 +26,9 @@ func Default() Config { return Config{Project: "siding", Container: "siding-proxy", Network: "siding", Image: Image, DefaultPort: 80} } +// AccessLogFlag is in the command of a proxy that records its requests. +const AccessLogFlag = "--accesslog=true" + // RoutesMount is where the proxy reads route files that siding ui writes. const RoutesMount = "/etc/siding/routes" @@ -41,6 +44,8 @@ type Settings struct { Budget int64 // Builds is the number of build slots, 0 for the default. Builds int + // Idle is the idle threshold in seconds, 0 for off, nil for the default. + Idle *int64 // RoutesDir is the directory on this machine with the route files of siding ui. RoutesDir string // HostAlias is what host.docker.internal resolves to in the proxy: the proxy network's @@ -50,7 +55,8 @@ type Settings struct { // ComposeFile renders the proxy's compose file. The constraint keeps Traefik to containers // siding labelled for this proxy's network, so foreign Traefik labels on the same daemon are -// never exposed. +// never exposed. The access log on stdout records only the router and the time of a request; +// Docker keeps it at 30 MiB at most (phase 4 design D3). func ComposeFile(c Config, s Settings) []byte { constraint := fmt.Sprintf("LabelRegex(`dev.siding.repo`, `.+`) && Label(`traefik.docker.network`, `%s`)", c.Network) labels := fmt.Sprintf(" dev.siding.machine: %q\n dev.siding.host-alias: %q\n", s.Machine, s.HostAlias) @@ -60,6 +66,9 @@ func ComposeFile(c Config, s Settings) []byte { if s.Builds > 0 { labels += fmt.Sprintf(" dev.siding.builds: \"%d\"\n", s.Builds) } + if s.Idle != nil { + labels += fmt.Sprintf(" dev.siding.idle: \"%d\"\n", *s.Idle) + } var b strings.Builder fmt.Fprintf(&b, `# Written by siding setup. Do not edit: siding setup rewrites it. services: @@ -76,6 +85,17 @@ services: - --providers.file.directory=%s - --providers.file.watch=true - --ping=true + - --accesslog=true + - --accesslog.format=json + - --accesslog.fields.defaultmode=drop + - --accesslog.fields.names.RouterName=keep + - --accesslog.fields.names.StartUTC=keep + - --accesslog.fields.headers.defaultmode=drop + logging: + driver: json-file + options: + max-size: 10m + max-file: "3" ports: - "127.0.0.1:%d:80" extra_hosts: diff --git a/internal/proxy/proxy_test.go b/internal/proxy/proxy_test.go new file mode 100644 index 0000000..c6e4e6d --- /dev/null +++ b/internal/proxy/proxy_test.go @@ -0,0 +1,26 @@ +package proxy + +import ( + "path/filepath" + "testing" + + "github.com/tweinmann/siding/internal/testutil" +) + +func TestComposeFile(t *testing.T) { + t.Parallel() + idle := int64(900) + for _, tt := range []struct { + name string + s Settings + }{ + {name: "defaults", s: Settings{Port: 80, Machine: "not-a-real-machine", RoutesDir: "/home/u/.siding/proxy/siding/routes", HostAlias: "172.19.0.1"}}, + {name: "settings", s: Settings{Port: 8080, Machine: "not-a-real-machine", Budget: 12 << 30, Builds: 2, Idle: &idle, + RoutesDir: "/home/u/.siding/proxy/siding/routes", HostAlias: HostGateway}}, + } { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + testutil.Golden(t, filepath.Join("testdata", "compose-"+tt.name+".yml"), ComposeFile(Default(), tt.s)) + }) + } +} diff --git a/internal/proxy/testdata/compose-defaults.yml b/internal/proxy/testdata/compose-defaults.yml new file mode 100644 index 0000000..efd2ab1 --- /dev/null +++ b/internal/proxy/testdata/compose-defaults.yml @@ -0,0 +1,46 @@ +# Written by siding setup. Do not edit: siding setup rewrites it. +services: + traefik: + image: "traefik:v3.7.16" + container_name: "siding-proxy" + restart: unless-stopped + command: + - --entrypoints.web.address=:80 + - --providers.docker=true + - --providers.docker.exposedbydefault=false + - --providers.docker.network=siding + - "--providers.docker.constraints=LabelRegex(`dev.siding.repo`, `.+`) && Label(`traefik.docker.network`, `siding`)" + - --providers.file.directory=/etc/siding/routes + - --providers.file.watch=true + - --ping=true + - --accesslog=true + - --accesslog.format=json + - --accesslog.fields.defaultmode=drop + - --accesslog.fields.names.RouterName=keep + - --accesslog.fields.names.StartUTC=keep + - --accesslog.fields.headers.defaultmode=drop + logging: + driver: json-file + options: + max-size: 10m + max-file: "3" + ports: + - "127.0.0.1:80:80" + extra_hosts: + - "host.docker.internal:172.19.0.1" + volumes: + - /var/run/docker.sock:/var/run/docker.sock:ro + - "/home/u/.siding/proxy/siding/routes:/etc/siding/routes:ro" + networks: [proxy] + labels: + dev.siding.machine: "not-a-real-machine" + dev.siding.host-alias: "172.19.0.1" + healthcheck: + test: ["CMD", "traefik", "healthcheck", "--ping"] + interval: 2s + timeout: 2s + retries: 30 +networks: + proxy: + name: "siding" + external: true diff --git a/internal/proxy/testdata/compose-settings.yml b/internal/proxy/testdata/compose-settings.yml new file mode 100644 index 0000000..3c935c3 --- /dev/null +++ b/internal/proxy/testdata/compose-settings.yml @@ -0,0 +1,49 @@ +# Written by siding setup. Do not edit: siding setup rewrites it. +services: + traefik: + image: "traefik:v3.7.16" + container_name: "siding-proxy" + restart: unless-stopped + command: + - --entrypoints.web.address=:80 + - --providers.docker=true + - --providers.docker.exposedbydefault=false + - --providers.docker.network=siding + - "--providers.docker.constraints=LabelRegex(`dev.siding.repo`, `.+`) && Label(`traefik.docker.network`, `siding`)" + - --providers.file.directory=/etc/siding/routes + - --providers.file.watch=true + - --ping=true + - --accesslog=true + - --accesslog.format=json + - --accesslog.fields.defaultmode=drop + - --accesslog.fields.names.RouterName=keep + - --accesslog.fields.names.StartUTC=keep + - --accesslog.fields.headers.defaultmode=drop + logging: + driver: json-file + options: + max-size: 10m + max-file: "3" + ports: + - "127.0.0.1:8080:80" + extra_hosts: + - "host.docker.internal:host-gateway" + volumes: + - /var/run/docker.sock:/var/run/docker.sock:ro + - "/home/u/.siding/proxy/siding/routes:/etc/siding/routes:ro" + networks: [proxy] + labels: + dev.siding.machine: "not-a-real-machine" + dev.siding.host-alias: "host-gateway" + dev.siding.budget: "12884901888" + dev.siding.builds: "2" + dev.siding.idle: "900" + healthcheck: + test: ["CMD", "traefik", "healthcheck", "--ping"] + interval: 2s + timeout: 2s + retries: 30 +networks: + proxy: + name: "siding" + external: true diff --git a/internal/stack/identity.go b/internal/stack/identity.go index 9c292df..2b94329 100644 --- a/internal/stack/identity.go +++ b/internal/stack/identity.go @@ -19,6 +19,7 @@ const ( LabelGitDir = "dev.siding.gitdir" LabelBudget = "dev.siding.budget" LabelBuilds = "dev.siding.builds" + LabelIdle = "dev.siding.idle" LabelShared = "dev.siding.shared" // LabelHostAlias, on siding-proxy, is what host.docker.internal resolves to in it. LabelHostAlias = "dev.siding.host-alias" diff --git a/internal/verdict/verdict.go b/internal/verdict/verdict.go index 6526030..5b13976 100644 --- a/internal/verdict/verdict.go +++ b/internal/verdict/verdict.go @@ -17,6 +17,8 @@ const ( Starting = "Starting" Failed = "Failed" TimedOut = "TimedOut" + // Stopped: no container runs and none failed; siding up starts the stack again. + Stopped = "Stopped" ) // Reasons a service is not ready. Part of the --json contract. @@ -55,6 +57,8 @@ type Input struct { Timeout time.Duration // Seed is set when a seed command of this up failed; it decides the verdict. Seed *Failure + // Healthchecks are the services the compose file gives a healthcheck. + Healthchecks map[string]bool } // Service is the state of one service. @@ -120,6 +124,8 @@ func Evaluate(in Input) Result { switch { case in.Seed != nil: res.Verdict, res.Failure = Failed, in.Seed + case stopped(in, byService): + res.Verdict = Stopped case firstFailed != nil: res.Verdict, res.Failure = Failed, firstFailed case allReady: @@ -132,6 +138,36 @@ func Evaluate(in Input) Result { return res } +// stopCodes are the exit codes of a stop: 0, or SIGKILL (137) and SIGTERM (143) for a process +// that does not exit 0 on SIGTERM. +var stopCodes = map[int]bool{0: true, 137: true, 143: true} + +// stopped reports whether the whole stack was stopped, not crashed: every service has a +// container, none runs, every exit code is one of a stop, and a service with a route path or a +// healthcheck shows it was meant to keep running. A stack of one-off services that all exited 0 +// is done, not stopped. +func stopped(in Input, byService map[string][]docker.Container) bool { + if len(in.Containers) == 0 { + return false + } + for _, name := range in.Services { + if len(byService[name]) == 0 { + return false + } + } + for _, c := range in.Containers { + if c.State != "exited" || !stopCodes[c.ExitCode] || c.OOMKilled { + return false + } + } + for _, name := range in.Services { + if _, routed := in.Routes[name]; routed || in.Healthchecks[name] { + return true + } + } + return false +} + // evaluateService looks at every container of a service; the worst one decides. func evaluateService(in Input, name string, containers []docker.Container) (Service, outcome, *Failure) { svc := Service{Name: name} diff --git a/internal/verdict/verdict_test.go b/internal/verdict/verdict_test.go index 0fcad7f..3f37654 100644 --- a/internal/verdict/verdict_test.go +++ b/internal/verdict/verdict_test.go @@ -129,6 +129,46 @@ func TestEvaluate(t *testing.T) { } } +func TestStopped(t *testing.T) { + t.Parallel() + routes := map[string]Route{"web": {URL: "http://shop.main.localhost/", Port: 3000}} + for _, tt := range []struct { + name string + services []string + containers []docker.Container + healthchecks map[string]bool + verdict string + }{ + {name: "stopped to make room", services: []string{"db", "migrate", "web"}, + containers: []docker.Container{ct("db", "exited", "", 0), ct("migrate", "exited", "", 0), ct("web", "exited", "", 137)}, verdict: Stopped}, + {name: "stopped by hand, SIGTERM", services: []string{"web"}, + containers: []docker.Container{ct("web", "exited", "", 143)}, verdict: Stopped}, + {name: "a healthcheck but no route", services: []string{"db"}, healthchecks: map[string]bool{"db": true}, + containers: []docker.Container{ct("db", "exited", "", 0)}, verdict: Stopped}, + {name: "a crash is not a stop", services: []string{"api", "web"}, + containers: []docker.Container{ct("api", "exited", "", 1), ct("web", "exited", "", 137)}, verdict: Failed}, + {name: "killed for lack of memory", services: []string{"web"}, + containers: []docker.Container{{ID: "web", Labels: map[string]string{stack.LabelService: "web"}, State: "exited", ExitCode: 137, OOMKilled: true}}, verdict: Failed}, + {name: "only one-off services stay Ready", services: []string{"migrate"}, + containers: []docker.Container{ct("migrate", "exited", "", 0)}, verdict: Ready}, + // After docker compose up -d of a stopped stack every container runs again. + {name: "starting again is Starting", services: []string{"db", "web"}, + containers: []docker.Container{ct("db", "running", "starting", 0), ct("web", "running", "", 0)}, verdict: Starting}, + {name: "a missing service is Missing", services: []string{"db", "web"}, + containers: []docker.Container{ct("web", "exited", "", 0)}, verdict: Failed}, + {name: "created, never started", services: []string{"web"}, + containers: []docker.Container{ct("web", "created", "", 0)}, verdict: Starting}, + } { + res := Evaluate(Input{Services: tt.services, Containers: tt.containers, Routes: routes, Healthchecks: tt.healthchecks, Timeout: time.Minute}) + if res.Verdict != tt.verdict { + t.Errorf("%s: verdict %s, want %s (failure %+v)", tt.name, res.Verdict, tt.verdict, res.Failure) + } + if tt.verdict == Stopped && (res.Failure != nil || len(res.Services) != len(tt.services)) { + t.Errorf("%s: %+v", tt.name, res) + } + } +} + func TestSeedFailed(t *testing.T) { t.Parallel() seed := &Failure{Service: "db", Reason: SeedFailed, Hint: "the seed command failed", Output: []string{"ERROR: relation exists"}} diff --git a/justfile b/justfile index 312ec82..f9f9bea 100644 --- a/justfile +++ b/justfile @@ -27,7 +27,7 @@ test: # Rewrite golden files from the current code after an intended change # (every package with golden files; a package without the -update flag would fail on it) golden: - go test ./internal/cli ./internal/stack -update + go test ./internal/cli ./internal/stack ./internal/proxy -update # The page is rendered from stacklist.json and gc-dryrun.json, so write it once they are new. go test ./internal/cli -run TestRenderPage -update diff --git a/openspec/changes/phase-4-idle-and-wake/.openspec.yaml b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/.openspec.yaml similarity index 70% rename from openspec/changes/phase-4-idle-and-wake/.openspec.yaml rename to openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/.openspec.yaml index 0d31f8b..1bb6456 100644 --- a/openspec/changes/phase-4-idle-and-wake/.openspec.yaml +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/.openspec.yaml @@ -1,3 +1,2 @@ schema: spec-driven created: 2026-10-28 -skip_specs: true diff --git a/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/design.md b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/design.md new file mode 100644 index 0000000..372ea75 --- /dev/null +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/design.md @@ -0,0 +1,150 @@ +# Design + +## Context + +v0.3 (archived as `2026-10-29-phase-3-snapshots-and-queue`) has a memory budget that refuses or +queues a new stack, learned peaks, and no way to give back memory a stack holds but does not +use. The pieces this phase builds on: the proxy's compose file rendered by +`internal/proxy.ComposeFile` with settings as labels on `siding-proxy`, recreated by Compose +when the file differs; router names `-[-route]`; `verdict.Evaluate` as a +pure function; `checkBudget` and the queue loop in `up`. Traefik is pinned at v3.7.16. +Motivation and scope: see proposal.md; behaviour: see the delta specs. Constraints from +CLAUDE.md: Docker is the database, no background service, standard building blocks over custom +code, never lose work. + +### Spike result (task 1.1, 2026-10-29) +The first version of this change planned idle stop and wake on request with Sablier next to the +proxy. The spike ran it in the inner daemon with Traefik v3.7.16, plugin v1.4.0, Sablier 1.10.1, +a stack of `db` (healthcheck, ready after 3 s), `migrate` (exits 0) and `web` (routed, +`depends_on: db: condition: service_healthy`), session 15 s, `blocking` strategy: + +- Worked: Traefik keeps the route of a stopped container with `allownonrunning`; a request + opens a session; Sablier stops the group after the session (exit codes 137 and 0); a request + wakes it; an access log with only `RouterName` and `StartUTC`; `USR1` reopens a renamed log. +- **Start order not kept**: on wake, `db` and `web` started in the same second; `web` did not + wait for `db` to be healthy. Compose's `depends_on` is ignored. +- **One-off services run again on every request**: Sablier starts `migrate` whenever it is + not running, i.e. on every request while the stack is awake, not once per wake. +- **The first request after a wake is answered `302 Found`** (a redirect to the same URL) by + the plugin, not by the service; browsers follow it, `curl` without `-L` does not. +- **The plugin needs the network on every start of the proxy**: Traefik 3.7 refuses the + plugin without `hash` ("plugin integrity check failed"; the catalog's validation answers + 404), and with the archive cached and `hash` pinned it still calls `plugins.traefik.io` on + start and disables all plugins when it cannot reach it. +- **Files written by the proxy into `~/.siding/proxy/` become root-owned**, so siding running + as the user cannot remove them without a helper container. + +The gate applied; the maintainer decided to drop Sablier (D1). This design is the revision. + +## Goals / Non-Goals + +**Goals:** +- Memory from stacks nobody used for a while, taken only when a new stack needs it, never from + active stacks, and without losing anything. +- A stopped stack is visible as such (`Stopped`) and is started again the way Compose starts it. +- Nothing new runs: no container, no daemon, no file of siding's; the record of requests is the + proxy's own container log. + +**Non-Goals:** +- Stopping a stack after a time without requests while nothing else needs memory (it needs + something that runs all the time). +- Waking a stack on a request (see the spike); moves to "later" in the roadmap. +- Counting activity that does not pass the proxy (`docker compose exec`, a worker's jobs). +- A page for requests to a stopped stack. + +## Decisions + +### D1. No Sablier (maintainer's decision after the spike) +Making room is done by `up` alone, with `docker compose`; a stopped stack is started again by +`siding up`. Alternatives, asked: Sablier with restrictions (the user excludes one-off services +with a label and adds `restart: on-failure` for `depends_on`, siding downloads the plugin and +loads it as a local plugin to start offline: many footnotes, and still not Compose's order); +postponing the phase until Sablier keeps the order. Rejected before: siding's own wake service +(custom code, and a background service). The guardrails do not change. + +### D2. The idle threshold (maintainer's decision: on by default) +Making room is on for every user, with a threshold of 5 minutes. `setup --idle ` +(a Go duration, at least 10 seconds; the end-to-end test uses 10s) keeps it in seconds as the +label `dev.siding.idle` on `siding-proxy`; `--idle off` writes `dev.siding.idle=0`, which turns +it off; `--idle default` removes the label; without the flag the label is kept, as with +`--budget`. `Setup` gains `idle`: the duration (`5m0s`) or `off`. Alternative, asked: opt-in +with `setup --idle` (v0.3 behaviour until turned on). Stopping loses nothing, so on by default +costs a user at most a `siding up`. + +### D3. The record of requests: the proxy's access log on stdout +Traefik writes a JSON access log to stdout next to its own log +(`--accesslog=true --accesslog.format=json --accesslog.fields.defaultmode=drop +--accesslog.fields.names.RouterName=keep --accesslog.fields.names.StartUTC=keep +--accesslog.fields.headers.defaultmode=drop`), so a line holds the router, the time and +Traefik's `level`, `msg` and `time`, and no URL, header or address (seen in the spike). The +proxy's compose file sets `logging: {driver: json-file, options: {max-size: 10m, max-file: +"3"}}`, so Docker bounds it at 30 MiB whatever the daemon's default driver is. Alternatives: a +file under `~/.siding/proxy/` (root-owned on Linux, see the spike, and rotation needs `USR1`); +Traefik's metrics (request counters without a time; siding would have to sample them in the +background); Sablier's sessions (dropped with D1). + +### D4. The last request of a stack +`internal/idle` reads the proxy's container log through the Engine API (stdout only), keeps +the lines that parse as JSON with a `RouterName`, and maps `@docker` to the project of +the running stacks by the longest project name that is a prefix followed by `-`, so +`shop-main-x-web` belongs to `shop-main-x`, not `shop-main`. The last request of a stack is the +latest of: its latest line, the newest `StartedAt` of its containers, the proxy's `StartedAt` +(a new proxy has an empty record, and a new stack has not had its chance yet). A proxy whose +command has no `--accesslog=true` (one of v0.3) has no record: nothing counts as idle, and `up` +says once on stderr that `siding setup` turns making room on. siding's own probes count as +requests, on purpose: an agent polling `status` is using the stack. + +### D5. Making room (maintainer's decision: only idle stacks, longest idle first) +Where `up` checks the budget (`checkBudget`, and in the queue only the first in line), a stack +that does not fit triggers: with the threshold on and a record present, list the other running +or partial stacks whose last request is at least the threshold ago; sort them oldest first; +take the shortest prefix whose memory in use makes the new stack fit. If even all of them do +not, stop nothing. Otherwise stop each with `docker compose -p stop` (by project name, +no compose file needed, as `down`), print `stopped idle stack (no request for 20m, +1.0 GiB)` on stderr, read the memory again, and continue. `--force` skips this. In the queue +the log is read at most every 30 seconds, not on every two-second check. Alternatives, asked: +any other stack, least recently used first (could stop the stack another agent is testing); +none. + +### D6. The verdict `Stopped` (maintainer's decision on the name) +`verdict.Evaluate` returns `Stopped` before anything else when no container is running, +restarting or starting, every exit code is 0, 137 or 143, and at least one service has a route +path or a healthcheck (a stack of one-off services that all exited 0 stays `Ready`, as in +v0.3). siding cannot tell who stopped a stack, so a stack stopped by hand is `Stopped` too; +both are started with `siding up`. `status` evaluates the containers before it probes and +never probes a `Stopped` stack; `status --wait` ends on it like on `Ready`; it exits 2 with the +stack's `url` and the hint `siding up --wait`. `ps` keeps its state `stopped`. Alternative, +asked: `Idle`, with a state `idle` in `ps` (claims to know why the stack is stopped). + +### D7. The skill +The skill names `Stopped` under exit 2 with `siding up --wait --json` to start the stack again, +says that `siding status --json` holds the stack's `url` without starting anything, and that +only the addresses siding reports work, never `localhost:` from the compose file. It +does not mention `--idle`: the threshold is the user's decision. + +### D8. Labels, codes, documents, tests +New label `dev.siding.idle` (proxy), into the conventions of CLAUDE.md and +`openspec/config.yaml`. New verdict `Stopped`; `Setup` gains `idle`; no new error code. +Golden files: the proxy's compose file, `Status` with `Stopped`, `Setup` with `idle`. The +end-to-end test makes room with `--idle 10s` and a small budget. + +## Risks / Trade-offs + +- [A stack used without the proxy (tests through `docker compose exec`, a worker) counts as + idle and can be stopped under pressure] → only under pressure and after the threshold; its + next `status` says `Stopped` with the hint, and the skill tells the agent to run + `siding up --wait`; `--idle 30m` or `--idle off` for such setups; noted in the README. +- [A request to a stopped stack gets a plain 404] → `ps`, `ui` and `status` show it as stopped; + a page for it is a non-goal of this phase. +- [Reading up to 30 MiB of log on a full budget] → only when a stack does not fit, and at most + every 30 seconds in the queue. +- [Traefik changes the fields of its JSON access log] → pinned image; a golden of the proxy's + compose file and the end-to-end test read a real line. +- [`docker compose -p stop` without a compose file] → Compose acts on the project's + containers by label, as for `down`; checked in the end-to-end test. + +## Migration Plan + +The first `siding setup` of v0.4 recreates the proxy with the record (Compose sees the changed +file); running stacks keep running. Until then `up` makes no room and says so. No label, file or +document needs migrating. Rollback: `siding setup --idle off`. diff --git a/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/proposal.md b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/proposal.md new file mode 100644 index 0000000..c803541 --- /dev/null +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/proposal.md @@ -0,0 +1,60 @@ +# Proposal + +## Why + +Most stacks sit idle most of the time: a branch waits for review, an agent works on code, not on +the running app. Every idle stack still holds its memory, so the budget fills with stacks +nobody uses, and the next `up` waits or is refused. When a new stack needs the memory, stopping +stacks nobody requested for a while gives it back without losing anything. + +## What Changes + +- Under memory pressure, `siding up` first stops other stacks that had no request through the + proxy for the idle threshold (5 minutes), longest idle first, when that makes the new stack + fit; only then does it wait in line or refuse as before. Nothing is removed: containers are + stopped, volumes and networks stay. +- `siding setup --idle ` changes the threshold, `--idle off` turns making room off; + the setting is a label on `siding-proxy`, like the budget and the build slots. +- The proxy records, per request, only the router and the time, in its container log, which + Docker keeps at a bounded size. That record is where the last request of a stack comes from. +- A stopped stack has a verdict of its own: `siding status` reports `Stopped` (exit 2) with its + address and the hint `siding up --wait`, instead of a `TimedOut` or `Starting` that would make + an agent look for a bug. +- A stopped stack is started again by `siding up` in its worktree, in Compose's order; a request + to it does not start it. +- Agent skill: the verdict `Stopped` and how to start the stack again; `siding status --json` + gives the stack's `url` without starting anything; the agent never gives `localhost:` + from the compose file, because siding publishes no ports. + +Not in this phase, after the spike of task 1.1 (see design.md): stopping a stack after a time +without requests, and waking it on a request. Sablier, the standard building block for both, +ignores Compose's `depends_on` order on wake, starts one-off services again on every request, and +needs the network on every start of the proxy. + +## Capabilities + +### New Capabilities +- `idle-stop`: the idle threshold, the last request of a stack, stopping idle stacks to make + room, and starting a stopped stack again. + +### Modified Capabilities +- `proxy`: `setup` reports the idle threshold; the proxy records router and time per request. +- `stack-status`: the verdict `Stopped`; `status --wait` ends at once on it. +- `memory-budget`: `up` refuses only after idle stacks could not make room; stopped stacks use + nothing. +- `cli-contract`: `Stopped` exits 2. +- `agent-skill`: what the skill teaches about `Stopped`, the address from `status`, and no + `localhost:`. +- `dev-environment`: `just smoke-stacks` also covers making room. + +## Impact + +- Code: `internal/proxy` (access log to stdout with two fields, bounded `json-file` logging), + `internal/verdict` (`Stopped`), `internal/cli` (`setup --idle`, `status`, `up` making room), a + new `internal/idle` (last request per stack from the proxy's log), `internal/docker` (container + logs with timestamps, stopping a project), `internal/e2e`. +- Documents: `Setup` gains `idle`; `Status` can carry `verdict: Stopped`. Only additions. +- The proxy is recreated by the first `siding setup` of v0.4; running stacks keep running. +- No new dependency, image, container or file under `~/.siding`. +- CLAUDE.md, README and `openspec/config.yaml`: the label `dev.siding.idle`, the roadmap (wake on + request moves to "later"). diff --git a/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/agent-skill/spec.md b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/agent-skill/spec.md new file mode 100644 index 0000000..a3d81f8 --- /dev/null +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/agent-skill/spec.md @@ -0,0 +1,30 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: What the skill teaches +The skill SHALL tell the agent to run `siding up --wait --json` after a change that affects the +running app, to read the verdict and exit code, to fix the named service from the failure's +reason, hint and log tail, to report the address when Ready, and on exit 3 (`NoCapacity`) to +stop and tell the user. It MUST NOT tell the agent to use `--force`, `--volumes`, `--idle`, +`siding gc` or `siding share`. + +#### Scenario: The commands in the skill +- **WHEN** the printed skill is read +- **THEN** it names `siding up --wait --json`, `siding status --json`, `siding logs --json` and the exit codes 0, 1, 2 and 3, and none of `--force`, `--volumes`, `--idle`, `gc` or `share` + +## ADDED Requirements + +### Requirement: Stopped stacks and addresses in the skill +The skill SHALL say that `Stopped` means the stack was stopped, for instance to make room, and +that `siding up --wait --json` starts it again; that `siding status --json` gives the stack's +address without starting anything; and that only the addresses siding reports work, never +`localhost:` from the compose file. + +#### Scenario: A stopped stack +- **WHEN** the printed skill is read +- **THEN** it names the verdict `Stopped` with exit 2 and `siding up --wait --json` to start the stack again + +#### Scenario: The address +- **WHEN** the printed skill is read +- **THEN** it says that `siding status --json` holds the stack's `url`, and that `localhost:` from the compose file does not work under siding diff --git a/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/cli-contract/spec.md b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/cli-contract/spec.md new file mode 100644 index 0000000..6653752 --- /dev/null +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/cli-contract/spec.md @@ -0,0 +1,25 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: Exit codes +siding SHALL exit 0 on success and when a stack is Ready, 1 when a command fails or a stack is +Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`) or is +stopped (Stopped), 3 when `up` refuses for lack of memory or waited in line until `--timeout` +(NoCapacity), and 130 without a message when interrupted. + +#### Scenario: A verdict decides the exit code +- **WHEN** `siding status --wait` ends with the verdict Failed +- **THEN** it exits 1, and with Ready it exits 0 + +#### Scenario: Stopped +- **WHEN** `siding status` reports the verdict Stopped +- **THEN** it exits 2 + +#### Scenario: Interrupted +- **WHEN** the user presses Ctrl-C during `siding logs -f` +- **THEN** siding stops docker compose and exits 130 without an error message + +#### Scenario: Over budget +- **WHEN** `siding up --wait` waits in line for memory until `--timeout` is up +- **THEN** it exits 3 without starting a container diff --git a/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/dev-environment/spec.md b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/dev-environment/spec.md new file mode 100644 index 0000000..1c8f1c2 --- /dev/null +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/dev-environment/spec.md @@ -0,0 +1,26 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: Stack smoke test +`just smoke-stacks` SHALL run siding's own commands end to end against the inner daemon: +`setup`, `up --wait`, routing by host and path, a `Failed` verdict with log tail, `ps`, `down`, +the refusal of `down --volumes`, and an `up` that stops an idle stack to make room, which then +reports `Stopped` and starts again with `up --wait`. Its proxy, network and projects MUST carry +the prefix `siding-test-`, and its proxy MUST listen on 18080. + +#### Scenario: Stack smoke test in CI +- **WHEN** a pull request is opened or pushed to, or a commit is pushed to `main` +- **THEN** CI runs `just smoke-stacks` inside the devcontainer, against its own daemon + +#### Scenario: It leaves nothing behind +- **WHEN** `just smoke-stacks` ends, passing or failing +- **THEN** no container, network or volume whose name starts with `siding-test-` is left in the inner daemon, and `siding-proxy` and the network `siding` are untouched + +#### Scenario: Not part of level 1 +- **WHEN** `just test` runs without a Docker daemon +- **THEN** it passes, because the end-to-end tests only build with the `docker` build tag + +#### Scenario: Making room +- **WHEN** the end-to-end test runs with `--idle 10s` and a small budget, starts one stack, waits 15 seconds without a request, and starts a second that only fits without the first +- **THEN** the first is stopped and reports `Stopped`, the second is Ready, and `up --wait` in the first's worktree after `down` of the second makes it Ready again with its data diff --git a/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/idle-stop/spec.md b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/idle-stop/spec.md new file mode 100644 index 0000000..d495982 --- /dev/null +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/idle-stop/spec.md @@ -0,0 +1,85 @@ +# Spec Delta + +## Purpose + +Defines how stacks nobody uses give their memory back when a new stack needs it: which stacks +count as idle, how `up` stops them to make room, and how a stopped stack is started again. + +## ADDED Requirements + +### Requirement: Idle threshold +A stack SHALL count as idle when it had no request through the proxy for the idle threshold, +5 minutes unless `siding setup --idle ` (at least `10s`) sets another, kept as the +label `dev.siding.idle` (seconds) on `siding-proxy`. `--idle off` SHALL turn making room off, +`--idle default` SHALL remove the label, and a `setup` without `--idle` MUST keep it. + +#### Scenario: Default +- **WHEN** the proxy carries no `dev.siding.idle` +- **THEN** a stack without a request for 5 minutes counts as idle + +#### Scenario: Another threshold +- **WHEN** `siding setup --idle 15m` runs, and later `siding setup --port 8080` +- **THEN** `siding-proxy` carries `dev.siding.idle=900` after both + +#### Scenario: Off +- **WHEN** `siding setup --idle off` runs +- **THEN** `siding up` never stops another stack, and behaves as v0.3 + +#### Scenario: Too short +- **WHEN** `siding setup --idle 5s` runs +- **THEN** it fails with `InvalidArguments` and changes nothing + +### Requirement: Last request of a stack +The last request of a stack SHALL be the latest request through the proxy to any of its +addresses. When the proxy has no record of a request since the stack or the proxy last +started, the later of those two starts SHALL count as its last request. + +#### Scenario: Any address counts +- **WHEN** only `http://api.shop.feature-x.localhost` was requested 2 minutes ago +- **THEN** the last request of `shop-feature-x` is 2 minutes ago + +#### Scenario: Never requested +- **WHEN** `shop-feature-x` was started 20 minutes ago and never requested +- **THEN** its last request counts as 20 minutes ago + +#### Scenario: The proxy was recreated +- **WHEN** `siding setup --port 8080` recreated the proxy 1 minute ago +- **THEN** no stack counts as idle before the threshold has passed after that + +### Requirement: Idle stacks make room +When a new stack does not fit into the budget, `siding up` SHALL first stop other running +stacks that are idle, longest idle first, as many as the new stack needs, but only when +stopping them makes it fit. It SHALL stop them without removing anything and name every stack +it stopped on stderr. The worktree's own stack MUST NOT be stopped; `--force` stops none. + +#### Scenario: Room from an idle stack +- **WHEN** the budget is 4 GiB, `shop-main` (1 GiB, last request 20 minutes ago) and `shop-fix` (2.5 GiB, last request 1 minute ago) run, and the new stack needs 1 GiB +- **THEN** `siding up` stops the containers of `shop-main`, says so on stderr, and starts the new stack + +#### Scenario: Not enough even then +- **WHEN** stopping every idle stack would still not make the new stack fit +- **THEN** `siding up` stops nothing and waits in line with `--wait` or refuses with `NoCapacity` as before + +#### Scenario: In line +- **WHEN** `siding up --wait` waits in line and another stack becomes idle +- **THEN** the first in line stops it when that makes it fit, and starts + +#### Scenario: A proxy without the record +- **WHEN** `siding-proxy` was created by siding v0.3, so it records no requests +- **THEN** `siding up` stops nothing, says on stderr that `siding setup` turns making room on, and waits or refuses as before + +#### Scenario: Data is kept +- **WHEN** `siding up` stopped `shop-main` to make room +- **THEN** the volumes and networks of `shop-main` remain, and `siding up --wait` in its worktree starts it again with its data + +### Requirement: Starting a stopped stack again +A stack stopped to make room SHALL be started again by `siding up` in its worktree, in the +order Compose starts it. A request through the proxy to a stopped stack MUST NOT start it. + +#### Scenario: up starts it +- **WHEN** `shop-main` was stopped to make room and `siding up --wait` runs in its worktree +- **THEN** Compose starts its services in their `depends_on` order and the command ends with the verdict + +#### Scenario: A request does not +- **WHEN** a browser requests `http://shop.main.localhost` while `shop-main` is stopped +- **THEN** the proxy answers 404 and the stack stays stopped diff --git a/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/memory-budget/spec.md b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/memory-budget/spec.md new file mode 100644 index 0000000..ad3bb13 --- /dev/null +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/memory-budget/spec.md @@ -0,0 +1,30 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: up refuses over budget +`siding up` SHALL refuse with `NoCapacity` and exit 3, before starting anything, when the +memory in use by every other siding stack on the machine plus the estimate of this stack +exceeds the budget and stopping idle stacks cannot make it fit (see the capability +`idle-stop`). The message MUST name the budget, the use and the estimate. `--force` starts it +anyway, without stopping any stack. + +#### Scenario: Over budget +- **WHEN** the budget is 4 GiB, other stacks use 3.5 GiB and the new stack's estimate is 768 MiB +- **THEN** `siding up` exits 3 with `NoCapacity` and the hint names `siding gc`, `siding down` in another worktree, `siding setup --budget` and `--force` + +#### Scenario: Forced +- **WHEN** `siding up --force` runs in the same situation +- **THEN** the stack starts, and a warning on stderr says the budget is exceeded + +#### Scenario: The stack itself does not count twice +- **WHEN** `siding up` runs again for a stack that already runs +- **THEN** only the other stacks' memory and this stack's estimate count, not its current use as well + +#### Scenario: Within budget +- **WHEN** the other stacks and the estimate fit into the budget +- **THEN** `siding up` behaves as without a budget + +#### Scenario: Stopped stacks use nothing +- **WHEN** a stack held 2 GiB before it was stopped to make room +- **THEN** it counts 0 bytes against the budget while it is stopped diff --git a/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/proxy/spec.md b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/proxy/spec.md new file mode 100644 index 0000000..d8fdb39 --- /dev/null +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/proxy/spec.md @@ -0,0 +1,41 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: Setup starts the proxy +`siding setup` SHALL create the network `siding` if it is missing and run the Compose project +`siding` with one container `siding-proxy`: Traefik with the Docker provider on the network +`siding`, restarting unless stopped. It SHALL print the proxy's address, budget, number of +build slots and idle threshold, with `--json` as a `Setup` document. + +#### Scenario: First setup +- **WHEN** `siding setup` runs on a machine without the proxy +- **THEN** the network `siding` and the running container `siding-proxy` exist, and it prints `http://*.localhost` as the address + +#### Scenario: JSON +- **WHEN** `siding setup --json` runs +- **THEN** it prints a `Setup` document with the proxy's container name, network, image, port, budget in bytes, build slots and `idle` (`5m0s`, another duration, or `off`) + +#### Scenario: Build slots +- **WHEN** `siding setup --builds 2` runs, and later `siding setup` without `--builds` +- **THEN** `siding-proxy` carries `dev.siding.builds=2` after both + +## ADDED Requirements + +### Requirement: Record of requests +The proxy SHALL record, for every request it routes, only the router and the time, in its +container log, which Docker keeps at a bounded size. The record MUST NOT hold the URL, query, +headers, cookies or client address of a request. It is losable: a recreated proxy starts an +empty record. + +#### Scenario: What a request leaves +- **WHEN** a browser requests `http://shop.main.localhost/account?token=not-a-real-token` with a cookie +- **THEN** the proxy's log holds one line with the router of `shop-main` and the time, and neither the path, the query nor the cookie + +#### Scenario: Bounded +- **WHEN** the proxy has routed many requests +- **THEN** its container log stays below 30 MiB + +#### Scenario: A proxy from siding v0.3 +- **WHEN** `siding-proxy` was created by siding v0.3 and `siding setup` of v0.4 runs +- **THEN** the proxy is recreated with the record, on the port it had, and running stacks keep running diff --git a/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/stack-status/spec.md b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/stack-status/spec.md new file mode 100644 index 0000000..261bc0c --- /dev/null +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/specs/stack-status/spec.md @@ -0,0 +1,58 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: Verdict +`siding status` SHALL report one verdict for the worktree's stack: `Ready`, `Starting`, +`Failed`, `TimedOut` or `Stopped`, the state of every service, and the stack's address. +Without `--wait` it SHALL report the current state once. + +#### Scenario: Status without wait +- **WHEN** `siding status` runs while a healthcheck is still starting +- **THEN** it reports `Starting` and exits 2 + +#### Scenario: No stack +- **WHEN** `siding status` runs in a worktree whose project has no containers +- **THEN** it fails with `StackNotFound` and the hint `siding up` + +### Requirement: Waiting +`siding status --wait` SHALL check until the verdict is `Ready`, `Failed` or `Stopped`, at most +for `--timeout` (default 3 minutes). When the time is up it SHALL report `TimedOut`, naming the +first service that is not ready, with reason, hint and log tail: `NotAnswering` for a routed +service that never answered, `NotReady` for one that is still starting. + +#### Scenario: Ready in time +- **WHEN** `siding status --wait --timeout 60s` runs and the stack becomes ready after 20 seconds +- **THEN** it reports `Ready` after about 20 seconds and exits 0 + +#### Scenario: Too slow +- **WHEN** `siding status --wait --timeout 10s` runs and `web` with route `/` never answers +- **THEN** it reports `TimedOut` with service `web`, reason `NotAnswering`, and exits 2 + +#### Scenario: Stopped +- **WHEN** `siding status --wait` runs while every container of the stack is stopped +- **THEN** it reports `Stopped` at once and exits 2, and starts nothing + +## ADDED Requirements + +### Requirement: Stopped +A stack SHALL be `Stopped` when none of its containers runs or restarts, every exit code is 0, +137 or 143 (the codes of a stop), and at least one of its services has a route path or a +healthcheck. The `Status` document SHALL carry the stack's address and the hint +`siding up --wait`. `status` MUST NOT probe a `Stopped` stack. + +#### Scenario: Stopped to make room +- **WHEN** `siding up` in another worktree stopped `shop-main` to make room, and `siding status --json` runs in the worktree of `shop-main` +- **THEN** it prints a `Status` document with `verdict: Stopped`, the stack's `url` and the hint, and exits 2 + +#### Scenario: Stopped by hand +- **WHEN** the user ran `docker compose -p shop-main stop` +- **THEN** the verdict is `Stopped` as well + +#### Scenario: A crash is not a stop +- **WHEN** every container of a stack is stopped and `api` exited with code 1 +- **THEN** the verdict is `Failed` with service `api`, reason `Exited` + +#### Scenario: Only one-off services +- **WHEN** a stack has no route path and no healthcheck, and every container exited 0 +- **THEN** the verdict is `Ready`, as before diff --git a/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/tasks.md b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/tasks.md new file mode 100644 index 0000000..9b4cd53 --- /dev/null +++ b/openspec/changes/archive/2026-10-29-phase-4-idle-and-wake/tasks.md @@ -0,0 +1,44 @@ +# Tasks + +## 1. Spike: Sablier against the pinned Traefik (gate) + +- [x] 1.1 In the inner daemon, run Traefik v3.7.16 with the plugin `v1.4.0` and `sablierapp/sablier:1.10.1` by hand (a `siding-test-spike` project, proxy on 18080) with a two-service stack (`web` routed, `db` with a healthcheck, `web` with `depends_on: condition: service_healthy`, plus a one-off service that exits 0); check and write into design.md: sleep after `sessionDuration`, wake by request with the `blocking` strategy, the start order within the group, what happens to the exited one-off service, what the blocking timeout answers, that a 1-second request opens a session, that a restart with the plugin cache and no network loads the plugin, Traefik's log line when it cannot, and `USR1` reopening the access log; remove the project by name. If the start order is not kept, stop and ask the maintainer (design, Risks) + - Result: the start order is not kept; the maintainer dropped Sablier (design D1), and this change was revised. + +## 2. Foundations + +- [x] 2.1 Add the verdict `Stopped` as a constant and the label `dev.siding.idle` to the conventions in CLAUDE.md and `openspec/config.yaml`, and move "wake on request" to "Later" in the roadmap there; verify `openspec validate --all --strict` passes +- [x] 2.2 Add to `internal/docker`: a container's stdout log as lines, and its command and `StartedAt`; add stopping a project by name (`docker compose -p stop`) next to `down`; verify with unit tests of the mappings and the compose arguments + +## 3. The proxy's record and the threshold + +- [x] 3.1 Add the JSON access log on stdout with only `RouterName` and `StartUTC` and the bounded `json-file` logging to `proxy.ComposeFile` (D3); verify with the updated golden of the proxy's compose file +- [x] 3.2 Add `setup --idle |off|default` with the label carried forward and the minimum of 10 seconds (D2), and `idle` in `Setup`; verify with fake-based tests per scenario of `idle-stop` "Idle threshold" and the updated `setup.json` golden +- [x] 3.3 Implement `internal/idle` (D4): parse the proxy's log lines, map routers to projects by longest prefix, the last request per stack as the latest of record, stack start and proxy start, and "no record" for a proxy without `--accesslog=true`; verify with table tests (prefix clash `shop-main` / `shop-main-x`, Traefik's own non-JSON lines, never requested, proxy recreated, a v0.3 proxy) + +## 4. Making room + +- [x] 4.1 Make room in `checkBudget` and for the first in line (D5): candidates, oldest first, shortest prefix that fits, nothing when it would not fit, stop by project name, the stderr line, `--force` and `--idle off` skip it, the hint for a proxy without the record, the log read at most every 30 seconds in the queue; verify with fake-based tests per scenario of `idle-stop` "Idle stacks make room" and `memory-budget` "Stopped stacks use nothing", using the fake clock +- [x] 4.2 Document making room in the README's budget section (threshold, what counts as a request, `--idle`, starting a stopped stack with `siding up`, activity that does not pass the proxy); verify the shown commands run as written in the devcontainer + +## 5. The verdict Stopped + +- [x] 5.1 Implement `Stopped` in `verdict.Evaluate` (D6); verify with verdict table tests (stopped to make room, stopped by hand, a crash is not a stop, only one-off services stay `Ready`, starting again is `Starting`) +- [x] 5.2 Make `status` evaluate before probing, never probe a `Stopped` stack, end `--wait` on it with exit 2, the `url` and the hint; verify with fake-based tests that no probe is sent and a `status-stopped.json` golden + +## 6. Agent skill + +- [x] 6.1 Update `internal/cli/skill/SKILL.md` (D7): `Stopped` under exit 2 and how to start the stack again, the `url` from `siding status --json`, never `localhost:`; verify with the skill test for the named commands and absent flags (`--idle` included) and the `skill.json` golden + +## 7. End-to-end and integration + +- [x] 7.1 Extend `internal/e2e`: `setup --idle 10s` with a small budget, one stack Ready, 15 seconds without a request, a second `up --wait` that stops the first and is Ready; the first reports `Stopped` and keeps its volume; a request to it answers 404; after `down` of the second, `up --wait` in the first's worktree makes it Ready with its data; the proxy's log line holds no path; verify with `just smoke-stacks` and that no `siding-test-*` object is left +- [x] 7.2 Run the flow by hand in the devcontainer and on the Mac (`just build-darwin`): `setup --port 8080 --idle 1m` with a small budget, two worktrees, the second `up` stopping the first, `siding status` and `siding ui` showing it stopped, `siding up --wait` starting it again; record the result in this task + - Devcontainer (2026-10-29): passed. Two worktrees of one repository, `--idle 1m` and a budget that fits one stack: after a minute without a request, `up` in `feature/two` stopped `demo-main` and said so on stderr; `status` in `main` reported `Stopped` with the hint and exit 2; a request answered 404; `ps` and `siding ui` showed it `stopped` with 0 MiB; `up --wait` made it Ready again. Found: branches of one repository share learned peaks, so they fit far more tightly than their `mem_limit` suggests (correct; the end-to-end test uses two repositories); `status` showed "not answering" for a stopped stack it never probed, now left out. + - macOS with Docker Desktop (arm64, 2026-10-29): passed, 15 of 15 checks of `bin/mac-test/run.sh` (setup `--idle 1m` on port 18090, a budget for one stack, two worktrees): `up` in `feature/two` stopped `main` after a minute without a request and said so; `status` `Stopped` with exit 2 and the hint; a request answered 404; `ps` and `siding ui` showed it stopped; `up --wait` made it Ready again with its data; the proxy it started was removed again. +- [x] 7.3 Update the status in CLAUDE.md, the README and `openspec/config.yaml` to v0.4; verify `just test` passes + +## Workflow follow-up + +- Stop and show the result; wait for the maintainer's approval of Phase 4. +- `/opsx:archive` merges the deltas into `openspec/specs/`. diff --git a/openspec/changes/phase-4-idle-and-wake/proposal.md b/openspec/changes/phase-4-idle-and-wake/proposal.md deleted file mode 100644 index a008753..0000000 --- a/openspec/changes/phase-4-idle-and-wake/proposal.md +++ /dev/null @@ -1,25 +0,0 @@ -# Proposal - -> Draft. This change holds the roadmap text only and is marked `skip_specs: true`. -> `/opsx:propose` turns it into a full change: it writes the delta specs, `design.md` and -> `tasks.md`, revises this proposal, and removes `skip_specs` from `.openspec.yaml`. - -## Why - -Most stacks sit idle most of the time. Stopping them and waking them on request frees memory -without the developer noticing. - -## What Changes - -Roadmap v0.4: - -- Idle stop and wake on request (a standard building block such as Sablier). -- Stopping idle stacks when the budget is under pressure. - -## Capabilities - -To be decided at `/opsx:propose`. - -## Impact - -To be decided at `/opsx:propose`. diff --git a/openspec/config.yaml b/openspec/config.yaml index 9d53b81..2da5471 100644 --- a/openspec/config.yaml +++ b/openspec/config.yaml @@ -24,7 +24,8 @@ context: | Status: pre-alpha. Phase 0 (repository, devcontainer, OpenSpec, CI) and Phase 1 (v0.1: stacks per worktree, routing through siding-proxy, the verdict of status) are done; Phase 2 (v0.2: the overview siding ui, the memory budget, gc, lint, teardown) is done; Phase 3 - (v0.3: snapshots and seeding, the start queue, the build limit, learned peaks) is done. + (v0.3: snapshots and seeding, the start queue, the build limit, learned peaks) is done; + Phase 4 (v0.4: idle stacks make room under memory pressure, the verdict `Stopped`) is done. OpenSpec is the only planning source: `openspec/specs/` is what siding does today, `openspec/changes/` what it is about to do (the roadmap is the `phase--` changes), `openspec/changes/archive/` how it got here. CLAUDE.md holds the @@ -92,8 +93,9 @@ context: | service with several), `dev.siding.seed` (`volume|script|none`), `dev.siding.seed.command`. Volume label `dev.siding.shared`. Snapshot volumes: `dev.siding.snapshot`, `dev.siding.snapshot-volume`, `dev.siding.snapshot-created`. On `siding-proxy`: - `dev.siding.budget` (bytes), `dev.siding.builds` (build slots) and `dev.siding.host-alias` - (what `host.docker.internal` resolves to). The machine and + `dev.siding.budget` (bytes), `dev.siding.builds` (build slots), `dev.siding.idle` (idle + threshold in seconds, `0` for off) and `dev.siding.host-alias` (what `host.docker.internal` + resolves to). The machine and worktree labels exist from v0.1 so that remote hosts need no migration later. - Development and every test run in a devcontainer with its own Docker daemon (Docker-in-Docker); see the spec `dev-environment`. There is no Docker-outside-of-Docker @@ -108,10 +110,11 @@ context: | memory budget (refuse, exit 3), `just build-darwin` to try siding on the Mac. - v0.3 (phase-3): snapshots and `--seed`, queue for capacity, parallel build limit, learned peaks. - - v0.4 (phase-4): idle stop and wake on request, stopping idle stacks under pressure. + - v0.4 (phase-4): stopping idle stacks under memory pressure, the verdict `Stopped`. - v0.5 (phase-5): remote Docker host (Docker context), local forwarder for `*.localhost`, `HOST` column in `ps`. - - Later: HTTP errors from the proxy in the verdict, a shared server with a database per + - Later: idle stop after a time and wake on request (Sablier ignores `depends_on` on wake, + see the phase 4 spike), HTTP errors from the proxy in the verdict, a shared server with a database per branch, integrations with agent managers, scoped Docker access for agents. ## Rules for proposals diff --git a/openspec/specs/agent-skill/spec.md b/openspec/specs/agent-skill/spec.md index 361b7e4..9961877 100644 --- a/openspec/specs/agent-skill/spec.md +++ b/openspec/specs/agent-skill/spec.md @@ -18,9 +18,23 @@ stdout and write no file. With `--json` it SHALL print a `Skill` document holdin The skill SHALL tell the agent to run `siding up --wait --json` after a change that affects the running app, to read the verdict and exit code, to fix the named service from the failure's reason, hint and log tail, to report the address when Ready, and on exit 3 (`NoCapacity`) to -stop and tell the user. It MUST NOT tell the agent to use `--force`, `--volumes`, `siding gc` -or `siding share`. +stop and tell the user. It MUST NOT tell the agent to use `--force`, `--volumes`, `--idle`, +`siding gc` or `siding share`. #### Scenario: The commands in the skill - **WHEN** the printed skill is read -- **THEN** it names `siding up --wait --json`, `siding status --json`, `siding logs --json` and the exit codes 0, 1, 2 and 3, and none of `--force`, `--volumes`, `gc` or `share` +- **THEN** it names `siding up --wait --json`, `siding status --json`, `siding logs --json` and the exit codes 0, 1, 2 and 3, and none of `--force`, `--volumes`, `--idle`, `gc` or `share` + +### Requirement: Stopped stacks and addresses in the skill +The skill SHALL say that `Stopped` means the stack was stopped, for instance to make room, and +that `siding up --wait --json` starts it again; that `siding status --json` gives the stack's +address without starting anything; and that only the addresses siding reports work, never +`localhost:` from the compose file. + +#### Scenario: A stopped stack +- **WHEN** the printed skill is read +- **THEN** it names the verdict `Stopped` with exit 2 and `siding up --wait --json` to start the stack again + +#### Scenario: The address +- **WHEN** the printed skill is read +- **THEN** it says that `siding status --json` holds the stack's `url`, and that `localhost:` from the compose file does not work under siding diff --git a/openspec/specs/cli-contract/spec.md b/openspec/specs/cli-contract/spec.md index 6eee52c..41bae6e 100644 --- a/openspec/specs/cli-contract/spec.md +++ b/openspec/specs/cli-contract/spec.md @@ -54,14 +54,18 @@ it SHALL print `error: ` on stderr, followed by a hint line when there ### Requirement: Exit codes siding SHALL exit 0 on success and when a stack is Ready, 1 when a command fails or a stack is -Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`), 3 when -`up` refuses for lack of memory or waited in line until `--timeout` (NoCapacity), and 130 -without a message when interrupted. +Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`) or is +stopped (Stopped), 3 when `up` refuses for lack of memory or waited in line until `--timeout` +(NoCapacity), and 130 without a message when interrupted. #### Scenario: A verdict decides the exit code - **WHEN** `siding status --wait` ends with the verdict Failed - **THEN** it exits 1, and with Ready it exits 0 +#### Scenario: Stopped +- **WHEN** `siding status` reports the verdict Stopped +- **THEN** it exits 2 + #### Scenario: Interrupted - **WHEN** the user presses Ctrl-C during `siding logs -f` - **THEN** siding stops docker compose and exits 130 without an error message diff --git a/openspec/specs/dev-environment/spec.md b/openspec/specs/dev-environment/spec.md index 67fcaf5..a4ac75b 100644 --- a/openspec/specs/dev-environment/spec.md +++ b/openspec/specs/dev-environment/spec.md @@ -98,9 +98,10 @@ port the compose file publishes, and that Traefik with the Docker provider on th ### Requirement: Stack smoke test `just smoke-stacks` SHALL run siding's own commands end to end against the inner daemon: -`setup`, `up --wait`, routing by host and path, a `Failed` verdict with log tail, `ps`, `down` -and the refusal of `down --volumes`. Its proxy, network and projects MUST carry the prefix -`siding-test-`, and its proxy MUST listen on 18080. +`setup`, `up --wait`, routing by host and path, a `Failed` verdict with log tail, `ps`, `down`, +the refusal of `down --volumes`, and an `up` that stops an idle stack to make room, which then +reports `Stopped` and starts again with `up --wait`. Its proxy, network and projects MUST carry +the prefix `siding-test-`, and its proxy MUST listen on 18080. #### Scenario: Stack smoke test in CI - **WHEN** a pull request is opened or pushed to, or a commit is pushed to `main` @@ -114,6 +115,10 @@ and the refusal of `down --volumes`. Its proxy, network and projects MUST carry - **WHEN** `just test` runs without a Docker daemon - **THEN** it passes, because the end-to-end tests only build with the `docker` build tag +#### Scenario: Making room +- **WHEN** the end-to-end test runs with `--idle 10s` and a small budget, starts one stack, waits 15 seconds without a request, and starts a second that only fits without the first +- **THEN** the first is stopped and reports `Stopped`, the second is Ready, and `up --wait` in the first's worktree after `down` of the second makes it Ready again with its data + ### Requirement: Test stacks Tests SHALL create stacks only as Compose projects whose name starts with `siding-test-`, and MUST remove them by their exact project name, also when the test fails. The smoke test's proxy diff --git a/openspec/specs/idle-stop/spec.md b/openspec/specs/idle-stop/spec.md new file mode 100644 index 0000000..2441d70 --- /dev/null +++ b/openspec/specs/idle-stop/spec.md @@ -0,0 +1,84 @@ +# idle-stop Specification + +## Purpose +Defines how stacks nobody uses give their memory back when a new stack needs it: which stacks +count as idle, how `up` stops them to make room, and how a stopped stack is started again. + +## Requirements + +### Requirement: Idle threshold +A stack SHALL count as idle when it had no request through the proxy for the idle threshold, +5 minutes unless `siding setup --idle ` (at least `10s`) sets another, kept as the +label `dev.siding.idle` (seconds) on `siding-proxy`. `--idle off` SHALL turn making room off, +`--idle default` SHALL remove the label, and a `setup` without `--idle` MUST keep it. + +#### Scenario: Default +- **WHEN** the proxy carries no `dev.siding.idle` +- **THEN** a stack without a request for 5 minutes counts as idle + +#### Scenario: Another threshold +- **WHEN** `siding setup --idle 15m` runs, and later `siding setup --port 8080` +- **THEN** `siding-proxy` carries `dev.siding.idle=900` after both + +#### Scenario: Off +- **WHEN** `siding setup --idle off` runs +- **THEN** `siding up` never stops another stack, and behaves as v0.3 + +#### Scenario: Too short +- **WHEN** `siding setup --idle 5s` runs +- **THEN** it fails with `InvalidArguments` and changes nothing + +### Requirement: Last request of a stack +The last request of a stack SHALL be the latest request through the proxy to any of its +addresses. When the proxy has no record of a request since the stack or the proxy last +started, the later of those two starts SHALL count as its last request. + +#### Scenario: Any address counts +- **WHEN** only `http://api.shop.feature-x.localhost` was requested 2 minutes ago +- **THEN** the last request of `shop-feature-x` is 2 minutes ago + +#### Scenario: Never requested +- **WHEN** `shop-feature-x` was started 20 minutes ago and never requested +- **THEN** its last request counts as 20 minutes ago + +#### Scenario: The proxy was recreated +- **WHEN** `siding setup --port 8080` recreated the proxy 1 minute ago +- **THEN** no stack counts as idle before the threshold has passed after that + +### Requirement: Idle stacks make room +When a new stack does not fit into the budget, `siding up` SHALL first stop other running +stacks that are idle, longest idle first, as many as the new stack needs, but only when +stopping them makes it fit. It SHALL stop them without removing anything and name every stack +it stopped on stderr. The worktree's own stack MUST NOT be stopped; `--force` stops none. + +#### Scenario: Room from an idle stack +- **WHEN** the budget is 4 GiB, `shop-main` (1 GiB, last request 20 minutes ago) and `shop-fix` (2.5 GiB, last request 1 minute ago) run, and the new stack needs 1 GiB +- **THEN** `siding up` stops the containers of `shop-main`, says so on stderr, and starts the new stack + +#### Scenario: Not enough even then +- **WHEN** stopping every idle stack would still not make the new stack fit +- **THEN** `siding up` stops nothing and waits in line with `--wait` or refuses with `NoCapacity` as before + +#### Scenario: In line +- **WHEN** `siding up --wait` waits in line and another stack becomes idle +- **THEN** the first in line stops it when that makes it fit, and starts + +#### Scenario: A proxy without the record +- **WHEN** `siding-proxy` was created by siding v0.3, so it records no requests +- **THEN** `siding up` stops nothing, says on stderr that `siding setup` turns making room on, and waits or refuses as before + +#### Scenario: Data is kept +- **WHEN** `siding up` stopped `shop-main` to make room +- **THEN** the volumes and networks of `shop-main` remain, and `siding up --wait` in its worktree starts it again with its data + +### Requirement: Starting a stopped stack again +A stack stopped to make room SHALL be started again by `siding up` in its worktree, in the +order Compose starts it. A request through the proxy to a stopped stack MUST NOT start it. + +#### Scenario: up starts it +- **WHEN** `shop-main` was stopped to make room and `siding up --wait` runs in its worktree +- **THEN** Compose starts its services in their `depends_on` order and the command ends with the verdict + +#### Scenario: A request does not +- **WHEN** a browser requests `http://shop.main.localhost` while `shop-main` is stopped +- **THEN** the proxy answers 404 and the stack stays stopped diff --git a/openspec/specs/memory-budget/spec.md b/openspec/specs/memory-budget/spec.md index 120c048..448438f 100644 --- a/openspec/specs/memory-budget/spec.md +++ b/openspec/specs/memory-budget/spec.md @@ -44,8 +44,9 @@ memory limit (`deploy.resources.limits.memory`, else `mem_limit`); without eithe ### Requirement: up refuses over budget `siding up` SHALL refuse with `NoCapacity` and exit 3, before starting anything, when the memory in use by every other siding stack on the machine plus the estimate of this stack -exceeds the budget. The message MUST name the budget, the use and the estimate. `--force` -starts it anyway. +exceeds the budget and stopping idle stacks cannot make it fit (see the capability +`idle-stop`). The message MUST name the budget, the use and the estimate. `--force` starts it +anyway, without stopping any stack. #### Scenario: Over budget - **WHEN** the budget is 4 GiB, other stacks use 3.5 GiB and the new stack's estimate is 768 MiB @@ -63,6 +64,10 @@ starts it anyway. - **WHEN** the other stacks and the estimate fit into the budget - **THEN** `siding up` behaves as without a budget +#### Scenario: Stopped stacks use nothing +- **WHEN** a stack held 2 GiB before it was stopped to make room +- **THEN** it counts 0 bytes against the budget while it is stopped + ### Requirement: Learned peaks Whenever siding reads the memory of running containers (`ps`, `ui`, `status`, `up`), it SHALL keep the highest use seen per repository and service in `~/.siding/peaks.json`. The file is a diff --git a/openspec/specs/proxy/spec.md b/openspec/specs/proxy/spec.md index 1dceafa..3c11754 100644 --- a/openspec/specs/proxy/spec.md +++ b/openspec/specs/proxy/spec.md @@ -9,8 +9,8 @@ on the network `siding`, started by `siding setup` and bound to the loopback int ### Requirement: Setup starts the proxy `siding setup` SHALL create the network `siding` if it is missing and run the Compose project `siding` with one container `siding-proxy`: Traefik with the Docker provider on the network -`siding`, restarting unless stopped. It SHALL print the proxy's address, budget and number of -build slots, with `--json` as a `Setup` document. +`siding`, restarting unless stopped. It SHALL print the proxy's address, budget, number of +build slots and idle threshold, with `--json` as a `Setup` document. #### Scenario: First setup - **WHEN** `siding setup` runs on a machine without the proxy @@ -18,7 +18,7 @@ build slots, with `--json` as a `Setup` document. #### Scenario: JSON - **WHEN** `siding setup --json` runs -- **THEN** it prints a `Setup` document with the proxy's container name, network, image, port, budget in bytes and build slots +- **THEN** it prints a `Setup` document with the proxy's container name, network, image, port, budget in bytes, build slots and `idle` (`5m0s`, another duration, or `off`) #### Scenario: Build slots - **WHEN** `siding setup --builds 2` runs, and later `siding setup` without `--builds` @@ -113,3 +113,21 @@ projects, and teardown names them. `--dry-run` removes nothing; with `--json` it #### Scenario: Snapshots - **WHEN** `siding teardown` runs without `--snapshots` and snapshots exist - **THEN** the snapshots stay and teardown names them; with `--snapshots` they are removed too + +### Requirement: Record of requests +The proxy SHALL record, for every request it routes, only the router and the time, in its +container log, which Docker keeps at a bounded size. The record MUST NOT hold the URL, query, +headers, cookies or client address of a request. It is losable: a recreated proxy starts an +empty record. + +#### Scenario: What a request leaves +- **WHEN** a browser requests `http://shop.main.localhost/account?token=not-a-real-token` with a cookie +- **THEN** the proxy's log holds one line with the router of `shop-main` and the time, and neither the path, the query nor the cookie + +#### Scenario: Bounded +- **WHEN** the proxy has routed many requests +- **THEN** its container log stays below 30 MiB + +#### Scenario: A proxy from siding v0.3 +- **WHEN** `siding-proxy` was created by siding v0.3 and `siding setup` of v0.4 runs +- **THEN** the proxy is recreated with the record, on the port it had, and running stacks keep running diff --git a/openspec/specs/stack-status/spec.md b/openspec/specs/stack-status/spec.md index 90d7110..857ce05 100644 --- a/openspec/specs/stack-status/spec.md +++ b/openspec/specs/stack-status/spec.md @@ -9,8 +9,8 @@ the last lines of its log. ### Requirement: Verdict `siding status` SHALL report one verdict for the worktree's stack: `Ready`, `Starting`, -`Failed` or `TimedOut`, the state of every service, and the stack's address. Without `--wait` -it SHALL report the current state once. +`Failed`, `TimedOut` or `Stopped`, the state of every service, and the stack's address. +Without `--wait` it SHALL report the current state once. #### Scenario: Status without wait - **WHEN** `siding status` runs while a healthcheck is still starting @@ -65,8 +65,8 @@ service, a reason (`Exited`, `Unhealthy`, `Restarting`, `Missing`, `NotAnswering - **THEN** the verdict is `Failed` with service `db`, reason `SeedFailed`, and the command's output as the log tail ### Requirement: Waiting -`siding status --wait` SHALL check until the verdict is `Ready` or `Failed`, at most for -`--timeout` (default 3 minutes). When the time is up it SHALL report `TimedOut`, naming the +`siding status --wait` SHALL check until the verdict is `Ready`, `Failed` or `Stopped`, at most +for `--timeout` (default 3 minutes). When the time is up it SHALL report `TimedOut`, naming the first service that is not ready, with reason, hint and log tail: `NotAnswering` for a routed service that never answered, `NotReady` for one that is still starting. @@ -78,6 +78,10 @@ service that never answered, `NotReady` for one that is still starting. - **WHEN** `siding status --wait --timeout 10s` runs and `web` with route `/` never answers - **THEN** it reports `TimedOut` with service `web`, reason `NotAnswering`, and exits 2 +#### Scenario: Stopped +- **WHEN** `siding status --wait` runs while every container of the stack is stopped +- **THEN** it reports `Stopped` at once and exits 2, and starts nothing + ### Requirement: Status document With `--json`, `status` SHALL print one `Status` document: the verdict, the stack (project, repository, branch, worktree, address), every service (state, health, route, address, @@ -86,3 +90,25 @@ answering), and for `Failed` and `TimedOut` the failure with service, reason, hi #### Scenario: An agent reads the failure - **WHEN** `siding status --wait --json` ends with `Failed` - **THEN** the document holds `verdict: Failed` and a `failure` with `service`, `reason`, `hint` and `logs`, and the process exits 1 + +### Requirement: Stopped +A stack SHALL be `Stopped` when none of its containers runs or restarts, every exit code is 0, +137 or 143 (the codes of a stop), and at least one of its services has a route path or a +healthcheck. The `Status` document SHALL carry the stack's address and the hint +`siding up --wait`. `status` MUST NOT probe a `Stopped` stack. + +#### Scenario: Stopped to make room +- **WHEN** `siding up` in another worktree stopped `shop-main` to make room, and `siding status --json` runs in the worktree of `shop-main` +- **THEN** it prints a `Status` document with `verdict: Stopped`, the stack's `url` and the hint, and exits 2 + +#### Scenario: Stopped by hand +- **WHEN** the user ran `docker compose -p shop-main stop` +- **THEN** the verdict is `Stopped` as well + +#### Scenario: A crash is not a stop +- **WHEN** every container of a stack is stopped and `api` exited with code 1 +- **THEN** the verdict is `Failed` with service `api`, reason `Exited` + +#### Scenario: Only one-off services +- **WHEN** a stack has no route path and no healthcheck, and every container exited 0 +- **THEN** the verdict is `Ready`, as before