Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,14 @@ changing architecture.
Phase 0 approved (2026-10-28). Phase 1 (v0.1) and Phase 2 (v0.2) approved and archived
(2026-10-28, `openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing` and
`…-phase-2-overview-and-budget`). Phase 3 (v0.3) approved and archived (2026-10-29,
`…/2026-10-29-phase-3-snapshots-and-queue`). siding today: `setup [--port] [--budget]
[--builds]`, `teardown [--snapshots]`, `up [--wait] [--force] [--seed]`, `status [--wait]`,
`ps`, `logs`, `open`, `down [--volumes]`, `snapshot save|ls|rm`, `gc`, `lint`, `ui`, `skill`;
routing by host and path through `siding-proxy`; a memory budget with a start queue and learned
peaks; build slots; `just smoke-stacks`, `just build-darwin`. Next: Phase 4
(`phase-4-idle-and-wake`, v0.4), starting with `/opsx:propose`.
`…/2026-10-29-phase-3-snapshots-and-queue`). Phase 4 (v0.4) approved and archived (2026-10-29,
`…/2026-10-29-phase-4-idle-and-wake`); its spike dropped Sablier (see its design.md). siding
today: `setup [--port] [--budget] [--builds] [--idle]`, `teardown [--snapshots]`, `up [--wait]
[--force] [--seed]`, `status [--wait]`, `ps`, `logs`, `open`, `down [--volumes]`,
`snapshot save|ls|rm`, `gc`, `lint`, `ui`, `skill`; routing by host and path through
`siding-proxy`; a memory budget with a start queue and learned peaks, where idle stacks make
room (the verdict `Stopped`); build slots; `just smoke-stacks`, `just build-darwin`. Next:
Phase 5 (`phase-5-remote-host`, v0.5), starting with `/opsx:propose`.

## Working agreements

Expand Down Expand Up @@ -139,8 +141,8 @@ requests and on `main`, all inside the devcontainer image.
label `dev.siding.shared`. Snapshot volumes: `dev.siding.snapshot` (name),
`dev.siding.snapshot-volume` (the stack's volume key), `dev.siding.snapshot-created`, plus
`dev.siding.repo` and `dev.siding.machine`. On `siding-proxy`: `dev.siding.budget` (the memory
budget in bytes), `dev.siding.builds` (build slots) and `dev.siding.host-alias` (what
`host.docker.internal` resolves to in the proxy).
budget in bytes), `dev.siding.builds` (build slots), `dev.siding.idle` (the idle threshold in seconds, `0`
for off) and `dev.siding.host-alias` (what `host.docker.internal` resolves to in the proxy).
- Compose project `<repo>-<branch-slug>`, host `<repo>.<branch-slug>.localhost`, proxy
`siding-proxy` on the network `siding`, override `.siding/override.yml`.
- Docker state is read through the Docker Engine API (Go SDK); lifecycle operations invoke
Expand Down
32 changes: 26 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
siding gives every git worktree its own Docker Compose stack, at its own `*.localhost` address,
and tells you and your coding agents whether it actually runs and what it costs.

> **Pre-alpha (v0.3).** Stacks per worktree, routing through one local proxy, the verdict of
> `siding status`, the overview `siding ui`, a memory budget with a queue and learned peaks,
> snapshots and seeding, a build limit, `gc` and `lint` work; expect
> **Pre-alpha (v0.4).** Stacks per worktree, routing through one local proxy, the verdict of
> `siding status`, the overview `siding ui`, a memory budget with a queue, learned peaks and
> idle stacks that make room, snapshots and seeding, a build limit, `gc` and `lint` work; expect
> rough edges and changes before v1. There are no releases
> yet: build it from source (see below). The plan is in [openspec/changes/](openspec/changes)
> (one change per phase), what siding does today in [openspec/specs/](openspec/specs).
Expand Down Expand Up @@ -52,14 +52,15 @@ ports, and puts siding's labels and routes on the services. `.siding/` ignores i
### Does it run?

```sh
siding status # Ready, Starting, Failed or TimedOut, per service
siding status --wait # until Ready or Failed, at most --timeout (3m)
siding status # Ready, Starting, Failed, TimedOut or Stopped, per service
siding status --wait # until Ready, Failed or Stopped, at most --timeout (3m)
siding ps # every stack on this machine: state, address, memory
```

`Ready` means every service with a healthcheck is healthy and every service with a route
answers through the proxy. A failure names the service, the reason, a hint and its last log
lines. Exit codes: 0 Ready, 1 Failed, 2 TimedOut or still starting. Every command prints one
lines. `Stopped` means no container runs and none failed; `siding up` starts it again. Exit
codes: 0 Ready, 1 Failed, 2 TimedOut, Stopped or still starting. Every command prints one
JSON document with `--json`, for scripts and agents.

### Before the first up: lint
Expand Down Expand Up @@ -143,6 +144,25 @@ began to wait. `siding ps` and `siding ui` show the line. A waiting `up` that yo
Ctrl-C, or that is killed, leaves the line. `siding ps`
shows the total against the budget.

Before a stack waits or is refused, idle stacks make room for it. `up` stops other stacks that
had no request through the proxy for 5 minutes, the one idle longest first, as many as the new
stack needs, and only when that makes it fit; it says on stderr which ones it stopped. Stopping
removes nothing: containers stop, volumes and networks stay. A stopped stack reports the
verdict `Stopped` (exit code 2), and `siding up` in its worktree starts it again, in Compose's
order; a request to its address does not start it (the proxy answers 404).

```sh
siding setup --idle 15m # another idle threshold; "default" is 5m
siding setup --idle off # never stop a stack to make room
```

A request is anything that passes the proxy, `siding status` checks included. Work that does
not, such as tests through `docker compose exec` or a queue worker, does not count, so such a
stack can be stopped while it is busy; raise the threshold or turn it off if that matters. The
proxy records only the router and the time of each request, in its container log, which Docker
keeps at 30 MiB; no URL, header or cookie is kept. After upgrading siding, run `siding setup`
once, so the proxy records requests: until then, no stack is stopped to make room.

### Overview

```sh
Expand Down
3 changes: 2 additions & 1 deletion internal/cli/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import (
)

// Exit codes. They are part of the contract with agents, like the --json documents: 0 Ready,
// 1 Failed, 2 TimedOut (or Starting without --wait), 3 NoCapacity. A command that fails with
// 1 Failed, 2 TimedOut or Stopped (or Starting without --wait), 3 NoCapacity. A command that fails with
// an Error document exits 1.
const (
exitOK = 0
Expand Down Expand Up @@ -95,6 +95,7 @@ type Docker interface {
VolumeSizes(ctx context.Context) (map[string]int64, error)
RunHelper(ctx context.Context, h docker.Helper) (exit int, output []string, err error)
Logs(ctx context.Context, id string, tail int) ([]string, error)
Stdout(ctx context.Context, id string) ([]string, error)
Memory(ctx context.Context, id string) (uint64, error)
}

Expand Down
7 changes: 7 additions & 0 deletions internal/cli/down.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package cli
import (
"context"
"fmt"
"io"
"sort"
"strings"

Expand Down Expand Up @@ -76,6 +77,12 @@ func (a *app) down(cmd *cobra.Command, f downFlags) (downDocument, error) {
return doc, nil
}

// stopProject stops every container of a project by name, without files and without removing
// anything; siding up starts it again.
func (a *app) stopProject(ctx context.Context, project string, stderr io.Writer) error {
return a.opts.Compose.Run(ctx, compose.Invocation{Dir: a.opts.Home, Project: project}, []string{"stop"}, stderr, stderr)
}

// keepsWork refuses to remove data while the worktree has changes or the branch is unmerged.
func (a *app) keepsWork(ctx context.Context, st existing) error {
dirty, err := a.opts.Git.Dirty(ctx, st.wt.Top)
Expand Down
14 changes: 14 additions & 0 deletions internal/cli/down_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -158,3 +158,17 @@ func TestDownOnlyData(t *testing.T) {
t.Errorf("status: exit %d, %s", code, stdout)
}
}

func TestStopProject(t *testing.T) {
t.Parallel()
e := newEnv(t)
a := &app{opts: e.opts}
if err := a.stopProject(t.Context(), "shop-main", &strings.Builder{}); err != nil {
t.Fatal(err)
}
runs := e.compose.recorded()
// By project name, without a compose file, and it removes nothing.
if len(runs) != 1 || runs[0].inv.Project != "shop-main" || len(runs[0].inv.Files) != 0 || !slices.Equal(runs[0].args, []string{"stop"}) {
t.Errorf("runs %+v", runs)
}
}
25 changes: 21 additions & 4 deletions internal/cli/env_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ func newEnv(t *testing.T) *env {
t: t,
base: base,
repo: filepath.Join(base, "shop"),
docker: &fakeDocker{logs: map[string][]string{}, memory: map[string]uint64{}, memTotal: 16 << 30},
docker: &fakeDocker{logs: map[string][]string{}, stdout: map[string][]string{}, memory: map[string]uint64{}, memTotal: 16 << 30},
compose: &fakeCompose{config: testConfig},
probe: &fakeProbe{answers: map[string]probe.Response{}},
browser: &fakeBrowser{},
Expand Down Expand Up @@ -114,6 +114,7 @@ func newEnv(t *testing.T) *env {
var publishedPort = regexp.MustCompile(`"127\.0\.0\.1:(\d+):80"`)
var budgetLabel = regexp.MustCompile(`dev\.siding\.budget: "(\d+)"`)
var buildsLabel = regexp.MustCompile(`dev\.siding\.builds: "(\d+)"`)
var idleLabel = regexp.MustCompile(`dev\.siding\.idle: "(\d+)"`)
var hostAliasLabel = regexp.MustCompile(`dev\.siding\.host-alias: "([^"]+)"`)

// composeProxy does what docker compose up does for the proxy's project: it recreates the
Expand Down Expand Up @@ -149,8 +150,16 @@ func (e *env) composeProxy(r composeRun) {
if m := hostAliasLabel.FindStringSubmatch(string(data)); m != nil {
labels[stack.LabelHostAlias] = m[1]
}
if m := idleLabel.FindStringSubmatch(string(data)); m != nil {
labels[stack.LabelIdle] = m[1]
}
var cmd []string
if strings.Contains(string(data), proxy.AccessLogFlag) {
cmd = []string{proxy.AccessLogFlag}
}
e.docker.proxy = &docker.Container{ID: fmt.Sprintf("proxy-%d", e.docker.proxyGen), Name: "siding-proxy", Image: proxy.Image,
State: "running", Labels: labels, Ports: []docker.Port{{HostIP: "127.0.0.1", HostPort: port, ContainerPort: 80}}}
State: "running", Labels: labels, Cmd: cmd, StartedAt: e.clock.Now(),
Ports: []docker.Port{{HostIP: "127.0.0.1", HostPort: port, ContainerPort: 80}}}
}

func (e *env) git(args ...string) {
Expand Down Expand Up @@ -237,8 +246,10 @@ type fakeDocker struct {
networks []docker.Network
volumes []docker.Volume
logs map[string][]string
memory map[string]uint64
ensured []string
// stdout is what each container wrote to stdout, e.g. the proxy's access log.
stdout map[string][]string
memory map[string]uint64
ensured []string
}

func (d *fakeDocker) add(c docker.Container) {
Expand Down Expand Up @@ -409,6 +420,12 @@ func (d *fakeDocker) Logs(_ context.Context, id string, tail int) ([]string, err
return lines, nil
}

func (d *fakeDocker) Stdout(_ context.Context, id string) ([]string, error) {
d.mu.Lock()
defer d.mu.Unlock()
return slices.Clone(d.stdout[id]), nil
}

func (d *fakeDocker) Memory(_ context.Context, id string) (uint64, error) {
d.mu.Lock()
defer d.mu.Unlock()
Expand Down
6 changes: 5 additions & 1 deletion internal/cli/json.go
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,8 @@ type setupDocument struct {
BudgetBytes int64 `json:"budgetBytes"`
// Builds is how many stacks may build images at once.
Builds int `json:"builds"`
// Idle is the idle threshold, e.g. 5m0s, or off.
Idle string `json:"idle"`
}

// stackDocument is what up prints without --wait.
Expand All @@ -136,11 +138,13 @@ type serviceAddress struct {
// statusDocument is the verdict of status and up --wait.
type statusDocument struct {
header
// Verdict is Ready, Starting, Failed or TimedOut.
// Verdict is Ready, Starting, Failed, TimedOut or Stopped.
Verdict string `json:"verdict"`
Stack stackRef `json:"stack"`
Services []serviceStatus `json:"services,omitempty"`
Failure *failureDocument `json:"failure,omitempty"`
// Hint says what to do about a Stopped stack.
Hint string `json:"hint,omitempty"`
}

type serviceStatus struct {
Expand Down
151 changes: 151 additions & 0 deletions internal/cli/makeroom.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
package cli

import (
"context"
"fmt"
"io"
"maps"
"slices"
"strings"
"time"

"github.com/tweinmann/siding/internal/budget"
"github.com/tweinmann/siding/internal/docker"
"github.com/tweinmann/siding/internal/idle"
"github.com/tweinmann/siding/internal/stack"
)

// roomInterval is how often a stack waiting in line reads the proxy's log to make room; the
// log can hold 30 MiB, too much for every check of the line.
const roomInterval = 30 * time.Second

// idleStack is a running stack nobody requested for the idle threshold.
type idleStack struct {
project string
since time.Time
memory int64
}

// roomMaker stops idle stacks when a new one does not fit (spec idle-stop, design D5). It says
// once per up when the proxy keeps no record of requests.
type roomMaker struct {
a *app
project string
stderr io.Writer
told bool
lastTry time.Time
}

// make stops the idle stacks, longest idle first, that the stack needs to fit, and reports
// whether it stopped any. It stops none when even all of them would not make it fit.
func (r *roomMaker) make(ctx context.Context, check budget.Check) (bool, error) {
a := r.a
r.lastTry = a.opts.Clock.Now()
proxyContainer, err := a.opts.Docker.Container(ctx, a.opts.Proxy.Container)
if err != nil || proxyContainer == nil {
return false, err
}
threshold, on := idleThreshold(proxyContainer)
if !on {
return false, nil
}
if !idle.Recording(proxyContainer.Cmd) {
if !r.told {
warnf(r.stderr, "the proxy keeps no record of requests yet, so no idle stack can make room; siding setup turns it on")
r.told = true
}
return false, nil
}
candidates, err := r.idleStacks(ctx, proxyContainer, threshold)
if err != nil {
return false, err
}
need := check.Used + check.Estimate - check.Budget
var freed int64
n := 0
for n < len(candidates) && freed < need {
freed += candidates[n].memory
n++
}
if freed < need {
return false, nil
}
now := a.opts.Clock.Now()
for _, s := range candidates[:n] {
if err := a.stopProject(ctx, s.project, r.stderr); err != nil {
return false, err
}
_, _ = fmt.Fprintf(r.stderr, "Stopped idle stack %s (no request for %s, %s) to make room for %s; siding up --wait in its worktree starts it again\n",
s.project, idleText(now.Sub(s.since)), bytesText(uint64(s.memory)), r.project)
}
return n > 0, nil
}

// due reports whether the line may read the proxy's log again.
func (r *roomMaker) due() bool {
return r.lastTry.IsZero() || !r.a.opts.Clock.Now().Before(r.lastTry.Add(roomInterval))
}

// idleStacks lists the other running stacks whose last request is at least threshold ago,
// longest idle first, with their memory in use.
func (r *roomMaker) idleStacks(ctx context.Context, proxyContainer *docker.Container, threshold time.Duration) ([]idleStack, error) {
a := r.a
cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelRepo: ""})
if err != nil {
return nil, err
}
byProject := map[string][]docker.Container{}
for _, c := range cs {
if p := c.Labels[stack.LabelProject]; p != "" && p != r.project {
byProject[p] = append(byProject[p], c)
}
}
lines, err := a.opts.Docker.Stdout(ctx, proxyContainer.ID)
if err != nil {
return nil, err
}
last := idle.LastRequests(lines, slices.Collect(maps.Keys(byProject)))
now := a.opts.Clock.Now()
var out []idleStack
for project, containers := range byProject {
var started time.Time
running := false
for _, c := range containers {
running = running || c.State == "running"
if c.StartedAt.After(started) {
started = c.StartedAt
}
}
since := idle.Since(last[project], started, proxyContainer.StartedAt)
if !running || now.Sub(since) < threshold {
continue
}
memory, err := a.memory(ctx, containers)
if err != nil {
return nil, err
}
s := idleStack{project: project, since: since}
for _, m := range memory {
s.memory += int64(m)
}
out = append(out, s)
}
slices.SortFunc(out, func(x, y idleStack) int {
if c := x.since.Compare(y.since); c != 0 {
return c
}
return strings.Compare(x.project, y.project)
})
return out, nil
}

// idleText is a duration without a request: 45s, 20m or 2h5m.
func idleText(d time.Duration) string {
switch {
case d < time.Minute:
return fmt.Sprintf("%ds", int(d/time.Second))
case d < time.Hour:
return fmt.Sprintf("%dm", int(d/time.Minute))
}
return fmt.Sprintf("%dh%dm", int(d/time.Hour), int(d%time.Hour/time.Minute))
}
Loading
Loading