diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 38d38f3..f574a4f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -34,7 +34,7 @@ jobs: strategy: fail-fast: false matrix: - smoke: [smoke-secrets, smoke-chart, smoke-init, smoke-apps] + smoke: [smoke-secrets, smoke-chart, smoke-init, smoke-apps, smoke-agent] name: cluster (${{ matrix.smoke }}) steps: - uses: actions/checkout@v7 diff --git a/CLAUDE.md b/CLAUDE.md index 9329e81..5a6f84a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,8 +1,11 @@ # shelf -A self-hosted app platform for a single Apple Silicon Mac mini. Tenants build images in their -own CI and push a deploy artifact to GHCR; shelf watches the artifact and deploys it. -Learning project for platform engineering — not for production. +A self-hosted GitOps platform for running parallel coding-agent branches as isolated, verified +previews on your own hardware, with a path to production through standard artefacts. Tenants +build images in their own CI and push a deploy artifact to GHCR; shelf watches the artifact and +deploys it. Target host: any Linux machine with k3s, with a one-step install on a Mac (Colima). +Pre-production: production is a separate cluster reached through eject (planned). Learning and +reference project for platform engineering. OpenSpec is the only planning source: what shelf does is in [openspec/specs/](openspec/specs), the roadmap and the work in progress are the changes in [openspec/changes/](openspec/changes), and @@ -75,7 +78,15 @@ dev environment and the host footprint became the spec `dev-environment`, and th draft changes (`phase-8-acceptance` for the open 8b/8d runs, `phase-9-…` to `phase-13-…`). Phase 8 closed (2026-10-27): release v0.6.0 published (binaries, chart `0.6.0`, platform `v0.6.0` on GHCR) and `phase-8-acceptance` archived. -Next: Phase 9 (Mac mini: host setup, Colima, `shelf doctor`, `shelf destroy`). +Roadmap re-cut (2026-10-27): shelf is repositioned as a platform for parallel coding-agent +branches as isolated, verified previews, and the core thesis comes before the host. New order: +9 agent interface, 10 branch previews, 11 verification chain, 12 hardening, 13 host (macOS with +Colima and Linux, formerly 9), 14 service and installer (formerly 10), 15 operations (formerly +12), 16 eject with Hetzner as the reference target. The wizard (formerly 11) moved to "Later" in +the context; the reference apps (formerly 13) became the acceptance of Phases 10 and 16. The Mac +lock-in is dropped; exposure stays the operator's decision; LAN access with certificates stays +rejected and previews are reached by port-forward. +Next: Phase 9 (agent interface: `--json`, `shelf app status --wait`, `shelf app open`). ## Working agreements @@ -97,19 +108,26 @@ Next: Phase 9 (Mac mini: host setup, Colima, `shelf doctor`, `shelf destroy`). ## Guardrails -- The registry is the only interface between build and platform. The platform never talks to - Git or forge APIs. That rule is about the cluster: the host service may call the GitHub API for +- The registry is the only interface between build and platform, also for previews. The + platform never talks to Git or forge APIs. That rule is about the cluster: the host service may call the GitHub API for convenience (checking for a newer release), as long as no deploy path depends on it. - Every action the admin UI offers exists as a CLI command, and both call the same function in `internal/ops`. The UI is a second face on one operations layer, never a second implementation. +- Exposure is an operator decision, made with `shelf app add` / `shelf app credentials` (CLI or + UI, audit-logged) and never part of `app.yaml`. A tenant repository, and so an agent working in + it, cannot make its app public. +- Agent-facing commands offer `--json` with stable fields, so an agent never parses text written + for people. +- The runtime output depends on no shelf-specific CRD: what runs in the cluster is Flux, Helm and + plain Kubernetes objects, so an app can be ejected to plain Flux and Helm. - The platform is generic: no knowledge of databases or specific services. Everything is a component. - Apps are isolated from each other. The only thing apps share is a connection the user assigned to several of them; namespace, secret values, tunnel and cloudflared are every app's own. - Prefer standard building blocks used by larger platforms (Flux, Helm, Traefik, Cloudflare Tunnel) over custom code. -- `shelf init cluster` must work against any kubecontext. No Colima or macOS assumptions - outside `internal/host`. +- `shelf init cluster` must work against any kubecontext. No Colima, macOS or Linux + distribution assumptions outside `internal/host`. ## Dev environment @@ -136,6 +154,7 @@ Mac. Tool versions are pinned in `.devcontainer/Dockerfile`. | `just smoke-chart` | devcontainer | install `examples/hello` with the chart, check the Phase 2 acceptance criteria (needs Docker Hub) | | `just smoke-init` | devcontainer | Phase 3 acceptance: init twice, re-push, routing and `stripPrefix` through Traefik (run `just cluster-reset` first; needs network) | | `just smoke-apps` | devcontainer | Phase 4: `shelf app add`/`rm`, rollout by polling, tampered artifact refused, restore from backup (needs network) | +| `just smoke-agent` | devcontainer | Phase 9: `--json` documents, `--wait --commit` (Ready only after the commit rolled out, Failed for an image that cannot be pulled, TimedOut for a commit that never arrives, Ready again for a fix after a failure; takes up to 10 minutes because Helm times out the broken upgrade first) and `shelf app open` with the app's Host and `stripPrefix` (needs Docker Hub) | | `just smoke-tenant ` | devcontainer | Phase 4 acceptance with a real tenant repo and GHCR; asks for the GHCR login, waits for a push | | `just smoke-expose ` | devcontainer | Phase 8b acceptance: the app gets the Cloudflare connection `smoke`, its own tunnel, DNS record and HTTPS, then is taken off again; asks for the Cloudflare API token (set `CF_ACCOUNT_ID` if it sees several accounts); needs a cluster with a host suffix | | `just smoke-quick [app]` | devcontainer | Phase 8d acceptance: `examples/hello` as `hello-quick` with `--quick` gets a quick tunnel, answers with its own Host, gets a new name after a restart, `--private`/`--quick`, rm; keeps the cluster's host suffix; opens a public trycloudflare.com tunnel for a few minutes | diff --git a/README.md b/README.md index 9726291..aa19f0d 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,12 @@ # shelf -**A small, self-hosted app platform for a single Apple Silicon Mac mini.** +**Build and verify on your hardware, ship to any cloud, with the same verified artefact.** + +shelf is a self-hosted GitOps platform for developers who let several coding agents work in +parallel. It runs on any Linux host with k3s, or with one command on a Mac (Colima). The goal is +that every branch gets its own isolated preview, only verified artefacts become reachable beyond +it, and the same verified state reaches any Kubernetes cloud as a pull request, without depending +on shelf. Describe your app in one short `app.yaml`, build your images in your own CI, and shelf runs them on Kubernetes: with persistent volumes, generated secrets and health checks. An app stays @@ -8,16 +14,20 @@ inside the cluster until you give it a public address: `https://. [!WARNING] -> shelf is a learning project for platform engineering. It is **not meant for production**, -> and it is **still under construction**: a push to your repository deploys your app, and an -> admin UI in the browser adds, changes and removes apps. The Mac mini setup, the background -> service, the installer and the setup wizard are still missing. See [Status](#status). - -Where this is going: shelf installs on the mini with one command, and everything after that -happens in an admin UI in your browser — adding an app, changing which version runs, removing it -again, later logs and resource usage. You do not need to know that Kubernetes is underneath. The -command line stays the expert interface for everything the UI can do, and the cluster stays open -to `kubectl` if you want it. +> shelf is a learning and reference project for platform engineering, and it is +> **pre-production**: production is a separate cluster reached through eject (planned). It is +> **still under construction**: today a push to your repository deploys your app, and the CLI or +> an admin UI in the browser adds, changes and removes apps. Branch previews, the verification +> chain, the host setup, the installer and eject are planned, not built. See [Status](#status). + +Where this is going: you run Claude Code (other agents later) in several worktrees at once. Every +branch your agents push gets its own preview of the complete stack, database included, in a +namespace of its own, reachable from your machine with `shelf app open` and from nowhere else. +Your agents push, wait for the rollout and read a machine-readable verdict. Only an artefact that +CI signed and scanned can be put on the internet, and the operator decides that, never the +repository. When an app is ready, `shelf eject` hands the same verified digest to your production +cluster as a pull request of plain Flux and Helm objects. shelf is for developers who are +willing to see Kubernetes; the cluster stays open to `kubectl`. ## Contents @@ -59,7 +69,7 @@ flowchart LR end B --> R[(GHCR)] I --> R - subgraph mini["Mac mini (k3s)"] + subgraph host["Your host (k3s)"] F[Flux] --> H[Helm chart shelf-app] H --> K[Deployments, StatefulSets,
Services, Ingress, PVCs] K --> X[Traefik] @@ -71,10 +81,10 @@ flowchart LR 1. Your workflow builds the images and runs `shelf render` on your `app.yaml`. This pins every image to a digest, fills in host names and ports, and checks your file. 2. It pushes the result to GHCR as a small deploy artifact. -3. Flux on the Mac mini notices the new artifact and hands it to the generic `shelf-app` Helm +3. Flux on your host notices the new artifact and hands it to the generic `shelf-app` Helm chart, which creates the Kubernetes objects. 4. Traefik routes requests by host name. An app you put on the internet gets a Cloudflare - Tunnel of its own, with cloudflared in its namespace, so no port on the mini is opened. + Tunnel of its own, with cloudflared in its namespace, so no port on the host is opened. What shelf does is specified in [openspec/specs/](openspec/specs); the roadmap and the work in progress are the changes in [openspec/changes/](openspec/changes). @@ -99,14 +109,19 @@ one starts. | 8 | Adding, changing and removing apps from the browser | ✅ done | | 8b | Connections instead of platform credentials: registry logins and Cloudflare accounts chosen per app, one tunnel per app | ✅ done | | 8d | Apps stay inside the cluster by default, quick tunnels, no cluster domain | ✅ done | -| 9 | Mac mini: host setup, Colima, `shelf doctor`, `shelf destroy` | planned | -| 10 | Service and one-command installer | planned | -| 11 | Setup wizard | planned | -| 12 | Logs, resource usage, self-update | planned | -| 13 | Reference apps | planned | +| 9 | Agent interface: `--json`, `shelf app status --wait`, `shelf app open` | planned | +| 10 | Branch previews: one isolated preview per branch, discovered from the registry | planned | +| 11 | Verification chain: signatures, scans, provenance; exposure only for verified digests | planned | +| 12 | Hardening: Pod Security, NetworkPolicies, smoke tests per preview | planned | +| 13 | Host: macOS with Colima and Linux with k3s, `shelf doctor`, `shelf destroy` | planned | +| 14 | Service and one-command installer | planned | +| 15 | Logs, resource usage, self-update | planned | +| 16 | Eject and promotion to a production cluster | planned | Phases 6 to 13 were re-cut on 2026-09-20, when the target product changed from a command-line -tool to an appliance with an admin UI. Every phase from here on is an +tool to an appliance with an admin UI. On 2026-10-27 the roadmap was re-cut again: shelf became +a platform for parallel agent branches, the previews and the verification chain moved ahead of +the host setup, the Mac lock-in was dropped, and the setup wizard was deferred. Every phase from here on is an [OpenSpec](https://github.com/Fission-AI/OpenSpec) change in [openspec/changes/](openspec/changes); the decisions and results of Phases 0 to 8d, with their reasons, are in [the baseline archive](openspec/changes/archive/2026-10-20-baseline). @@ -114,10 +129,9 @@ the decisions and results of Phases 0 to 8d, with their reasons, are in ## Quick start > [!NOTE] -> On the finished product this becomes one command on the mini — -> `curl -fsSL https://raw.githubusercontent.com/tweinmann/shelf/v0/install.sh | sh` — followed by -> a setup wizard in the browser. That installer arrives in Phase 10. Until then, install the -> binary yourself: +> On the finished product this becomes one command on the host — +> `curl -fsSL https://raw.githubusercontent.com/tweinmann/shelf/v0/install.sh | sh`. That +> installer arrives in Phase 14. Until then, install the binary yourself: Download the latest release for your platform (linux or darwin, amd64 or arm64): @@ -230,8 +244,14 @@ shelf app add oci://ghcr.io//:main --registry ghcr same artifact can be added under several names, for instance `shop` and `shop-staging`; each gets a namespace and secrets of its own. A public artifact needs no `--registry`. -The app now answers at `.shelf.internal`, inside the cluster only. To reach it from the -internet, choose one of two ways: +The app now answers at `.shelf.internal`, inside the cluster only. To look at it from a +browser on your machine, without putting it anywhere else: + +```sh +shelf app open --browser +``` + +To reach it from the internet, choose one of two ways: - **Your own domain** through your Cloudflare account. Define the connection once; the token needs Account:Cloudflare Tunnel:Edit and Zone:DNS:Edit, and stays on your machine in @@ -252,6 +272,11 @@ internet, choose one of two ways: takes the app off the internet again. From then on, every push to `main` reaches the cluster by itself, usually within two minutes. +A coding agent working in the repository waits for its own commit with +`shelf app status --wait --commit "$(git rev-parse HEAD)" --json` and reads the verdict +(see [Output for scripts and agents](#output-for-scripts-and-agents)); +[examples/tenant](examples/tenant) ships a Claude Code skill and a `CLAUDE.md` that explain the +loop. `shelf app rm ` removes the app with all its data, its tunnel and its DNS record. ## Writing an `app.yaml` @@ -331,7 +356,7 @@ stays as it is. So `sh -c 'echo $HOME'` works without escaping. ### Things to know -- **Volume sizes are not enforced.** The local storage on the Mac mini does not limit +- **Volume sizes are not enforced.** The local storage on the host does not limit capacity, so `size` is documentation. A component can fill the disk. - **Plan volumes up front.** Kubernetes does not allow changing the volumes of a StatefulSet in place, and the local storage cannot grow, so changing an existing volume later needs manual @@ -391,14 +416,59 @@ drift. | `shelf connection packages ` / `zones ` | Lists the deploy artifacts a registry connection can pull, or the zones of a Cloudflare connection's account. | | `shelf app add ` | Deploys an app from a deploy artifact under the name `` and keeps it updated. The name is independent of the package, so one artifact can be added under several names. `--registry` picks the registry connection; `--cloudflare` with `--domain` puts the app on the internet under its own domain, `--quick` through a quick tunnel; without either it stays inside the cluster. Generates the app's secrets, stores them in the cluster and in `~/.shelf/apps//secrets.yaml`, and restores them from there after a cluster rebuild. Refuses at once if the cluster has no platform. Waits until the app is ready. Safe to run again, e.g. after adding a secret. | | `shelf app credentials ` | Changes the app's connections or domain and deploys it again: `--registry`, `--no-registry`, `--cloudflare`, `--domain`, `--quick`, `--private`. | +| `shelf app open ` | Makes the app reachable from a browser on this machine at `http://127.0.0.1:/`, for as long as it runs: requests go through the Kubernetes API server to the cluster's Traefik with the app's own Host header, so routes and `stripPrefix` behave as through a tunnel. Works for apps that stay inside the cluster. Listens on 127.0.0.1 only; `--port` picks the port, `--browser` opens it. It is not an exposure and changes nothing about the app. | | `shelf app tags ` | Lists the tags of the app's deploy artifact, newest first. | | `shelf app redeploy ` | Fetches the app's artifact again under the tag it is registered with and rolls out what it finds. For a tag that moved, or a deploy worth another try. | | `shelf app secrets ` | Lists the names of the generated secrets; `--reveal` prints the values. | -| `shelf app status [app]` | Lists the apps with their state, or walks one app from its deploy artifact to its running pods and says which step is not ready. For one app it also names its components and where each of them answers. Same chain as the admin UI shows. | +| `shelf app status [app]` | Lists the apps with their state, or walks one app from its deploy artifact to its running pods and says which step is not ready. For one app it also shows the git commit the running artifact was built from, and names its components and where each of them answers. Same chain as the admin UI shows. `--wait` waits until the app is Ready or a step failed and prints the verdict; `--commit ` first waits until the app runs the artifact built from that commit. Exit status 0 Ready, 1 failed, 2 timed out (`--timeout`, default 10m). | | `shelf serve` | Runs the admin UI on port 7654 of the local network (`--listen 127.0.0.1:7654` keeps it off the network). The first start prints a setup code that claims the instance; after that a password protects it. | | `shelf app rm ` | Removes an app with its namespace, volumes, secrets, tunnel and DNS record, after asking. The secret backup stays. | | `shelf version` | Prints the version. | +### Output for scripts and agents + +`shelf app status`, `shelf app tags`, `shelf connection list` and `shelf version` take `--json` +and then print one JSON document on stdout and nothing else there: + +```console +$ shelf app status hello --json +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "AppStatus", + "name": "hello", + "phase": "Ready", + "artifact": "oci://ghcr.io/o/hello:main", + "commit": "42a964eb0b4bd7248305d5f22233d6dbabcd56f3", + "stages": [ ... ], + ... +} +``` + +Every document has `apiVersion` and `kind` (`AppList`, `AppStatus`, `WaitResult`, `TagList`, +`ConnectionList`, `Version`). A command that fails prints a document of kind `Error` with a +`message`, and exits with status 1. Within one `apiVersion`, no field is removed, renamed or +given another meaning; new fields may appear, so ignore the ones you do not know. Empty fields +are left out rather than `null`. `shelf app secrets` has no `--json`, so that secret values never +end up in an agent's transcript. + +An agent that pushed a commit waits for it like this: + +```console +$ shelf app status hello --wait --commit "$(git rev-parse HEAD)" +hello: Failed, running commit 42a964eb0b4b + the pods: ImagePullBackOff: Back-off pulling image + the image cannot be pulled; check the reference and whether the cluster may read that registry +$ echo $? +1 +``` + +The wait reads the commit from the deploy artifact the cluster holds, which the reusable workflow +stamps with the commit it was built from, and is Ready only once every step has caught up with +that artifact. It never changes anything: the platform notices a new artifact within a minute +by itself. With `--json` the verdict is a `WaitResult` with `outcome` (`Ready`, `Failed` or +`TimedOut`) and, unless Ready, the `stage` that holds the app up, with its `reason`, `message` +and `hint`. + Example of an error message: ```text @@ -406,7 +476,7 @@ app.yaml:12: error: components.web.route: route needs a port; add port or ports ``` `shelf service`, which installs the admin UI as a background service, and the commands for the -Mac mini (`shelf init host`, `shelf doctor`, `shelf destroy`) follow in later phases. +host (`shelf init host`, `shelf doctor`, `shelf destroy`) follow in later phases. ## Developing shelf @@ -434,7 +504,7 @@ Useful commands: | `just build` | Build `bin/shelf` | | `just cluster-up` / `cluster-stop` / `cluster-down` / `cluster-reset` | Manage the local k3d cluster `shelf-dev` | | `just platform-push`, `just chart-push`, `just init-cluster` | Push `platform/` and the chart to the local registry; `init-cluster` pushes both and installs them with `shelf init cluster` | -| `just smoke-secrets`, `just smoke-chart`, `just smoke-init`, `just smoke-apps` | Smoke tests against the local cluster, also run in CI: secret expansion, the chart, `shelf init cluster` with routing through Traefik, and `shelf app add`/`rm` with rollouts from the local registry | +| `just smoke-secrets`, `just smoke-chart`, `just smoke-init`, `just smoke-apps`, `just smoke-agent` | Smoke tests against the local cluster, also run in CI: secret expansion, the chart, `shelf init cluster` with routing through Traefik, `shelf app add`/`rm` with rollouts from the local registry, and the agent interface (`--json`, `--wait --commit`, `shelf app open`) | | `just smoke-registry `, `just smoke-tenant `, `just smoke-expose ` | Smoke tests that need credentials: a private image pull, a real tenant repository end to end, and an app's own Cloudflare tunnel (needs a cluster with a host suffix) | | `just smoke-quick` | A quick tunnel for `examples/hello`; puts it on the internet for a few minutes | | `hack/nuke.sh` | **Run in a terminal on your machine, not in the container.** Removes every Docker object shelf created. | diff --git a/examples/tenant/.claude/skills/shelf/SKILL.md b/examples/tenant/.claude/skills/shelf/SKILL.md new file mode 100644 index 0000000..e834262 --- /dev/null +++ b/examples/tenant/.claude/skills/shelf/SKILL.md @@ -0,0 +1,60 @@ +--- +name: shelf +description: Check that a pushed commit of this app runs on shelf, and find out why it does not. Use after every git push to main, and whenever the user asks whether the app is deployed, healthy or broken. +--- + +# Deploying this app with shelf + +This repository is a shelf tenant: CI builds the images, renders `app.yaml` and pushes a deploy +artifact; shelf notices it within a minute and rolls it out. You never deploy by hand. Your job +after a push is to wait for the verdict on *your* commit and fix what it names. + +The app's name on shelf is in `CLAUDE.md` (the operator chose it; it is not `name` in +`app.yaml`). Below it is written ``. + +## The loop + +1. Commit and push to `main`. +2. Wait for the verdict on exactly that commit: + + ```sh + shelf app status --wait --commit "$(git rev-parse HEAD)" --json + ``` + + This blocks until the app runs that commit and is Ready, a step failed, or 10 minutes + passed (`--timeout 20m` for a slow build). It only reads the cluster; never try to make it + faster. +3. Read the exit status and the `WaitResult` document on stdout: + + | Exit | `outcome` | What to do | + |---|---|---| + | 0 | `Ready` | Done. The commit runs. | + | 1 | `Failed` | Read `stage.name`, `stage.reason`, `stage.message` and `stage.hint`, fix the cause in this repository, commit, push, and wait again. | + | 2 | `TimedOut` | `stage` says where it waited. "waiting for commit …" means the artifact of your commit never arrived: check the CI run of the push first (`gh run list`, `gh run view --log-failed`). | + + After a failed commit, the next one can wait several minutes at "the app" with "waiting for + the release to pick up the new values": Helm first gives up on the broken upgrade and rolls it + back. That is expected; keep waiting. + + Any other failure prints a document of kind `Error` with a `message` (no cluster, no such + app) and exits 1 without a `WaitResult`; report it, it is not something to fix in code. + +4. For the whole picture of one app, run `shelf app status --json`: every stage from the + deploy artifact to the pods, with the running `commit`, the components and their addresses. + +## What the stages mean + +`the deploy artifact` (CI did not push, or the registry refuses the login) → `the app values` +(the rendered app.yaml was rejected) → `the chart` (platform problem; report it) → `the app` +(the Helm release; often a bad `app.yaml`) → `the pods` (image pull, crash loop, out of memory, +missing secret). The first stage that is not Ready is the cause; everything after it is a +consequence. `shelf validate app.yaml` catches most app.yaml mistakes before a push. + +## What you cannot do + +- Make the app public. Exposure (a domain or a tunnel) is the operator's decision, made with + `shelf app add` or `shelf app credentials` outside this repository. Nothing in `app.yaml` + changes it, so do not try. +- Read secret values. `shelf app secrets` is for the operator. +- `shelf app open ` makes the app reachable in a browser on the operator's machine. It is + for the person, not for you; ask them to look if a page has to be checked by eye. diff --git a/examples/tenant/CLAUDE.md b/examples/tenant/CLAUDE.md new file mode 100644 index 0000000..1e69741 --- /dev/null +++ b/examples/tenant/CLAUDE.md @@ -0,0 +1,13 @@ +# greeter + +A shelf tenant: `app.yaml` describes the app, `.github/workflows/deploy.yml` builds and +publishes it, and shelf deploys every push to `main`. + +## Deploying + +The app runs on shelf as **`greeter`**. (Operator: replace this with the name you gave it in +`shelf app add …`; it need not match `name` in `app.yaml`.) + +After every push to `main`, use the `shelf` skill: wait with +`shelf app status greeter --wait --commit "$(git rev-parse HEAD)" --json` and fix what the +verdict names. The work is done when that command exits 0. diff --git a/examples/tenant/README.md b/examples/tenant/README.md index 6ad497e..2ad1095 100644 --- a/examples/tenant/README.md +++ b/examples/tenant/README.md @@ -17,3 +17,16 @@ same artifact can run more than once: ```sh shelf app add greeter-staging oci://ghcr.io/tweinmann/greeter:main ``` + +## Coding agents + +`CLAUDE.md` and the skill in `.claude/skills/shelf/` tell Claude Code how to work with shelf in +this repository: after a push, wait for the verdict on that commit with + +```sh +shelf app status greeter --wait --commit "$(git rev-parse HEAD)" --json +``` + +and fix what it names. Put the name you gave the app in `shelf app add` into `CLAUDE.md`. The +agent cannot make the app public: exposure is set with `shelf app add` or +`shelf app credentials`, never in this repository. diff --git a/go.mod b/go.mod index 753b47b..1216336 100644 --- a/go.mod +++ b/go.mod @@ -29,11 +29,13 @@ require ( github.com/go-openapi/swag v0.23.0 // indirect github.com/google/gnostic-models v0.7.0 // indirect github.com/google/uuid v1.6.0 // indirect + github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/josharian/intern v1.0.0 // indirect github.com/json-iterator/go v1.1.12 // indirect github.com/klauspost/compress v1.19.2 // indirect github.com/mailru/easyjson v0.7.7 // indirect + github.com/moby/spdystream v0.5.1 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect @@ -59,6 +61,7 @@ require ( k8s.io/api v0.36.4 // indirect k8s.io/klog/v2 v2.140.0 // indirect k8s.io/kube-openapi v0.0.0-20260317180543-43fb72c5454a // indirect + k8s.io/streaming v0.36.4 // indirect k8s.io/utils v0.0.0-20260210185600-b8788abfbbc2 // indirect sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 // indirect sigs.k8s.io/randfill v1.0.0 // indirect diff --git a/go.sum b/go.sum index e5fd698..62d76ae 100644 --- a/go.sum +++ b/go.sum @@ -1,3 +1,5 @@ +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio= +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= github.com/bahlo/generic-list-go v0.2.0 h1:5sz/EEAK+ls5wF+NeqDpk5+iNdMDXrh3z3nPnH1Wvgk= github.com/bahlo/generic-list-go v0.2.0/go.mod h1:2KvAjgMlE5NNynlg/5iLrrCCZ2+5xWbdbCW3pNTGyYg= github.com/buger/jsonparser v1.1.2 h1:frqHqw7otoVbk5M8LlE/L7HTnIq2v9RX6EJ48i9AxJk= @@ -37,6 +39,8 @@ github.com/google/go-containerregistry v0.22.1/go.mod h1:bJR35SK8XgisYmhg/FMQ/5R github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 h1:JeSE6pjso5THxAzdVpqr6/geYxZytqFMBCOtn/ujyeo= +github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674/go.mod h1:r4w70xmWCQKmi1ONH4KIaBptdivuRPyosB9RmPlGEwA= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= github.com/invopop/jsonschema v0.14.0 h1:MHQqLhvpNUZfw+hM3AZDYK7jxO8FZoQeQM77g8iyZjg= @@ -56,6 +60,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= +github.com/moby/spdystream v0.5.1 h1:9sNYeYZUcci9R6/w7KDaFWEWeV4LStVG78Mpyq/Zm/Y= +github.com/moby/spdystream v0.5.1/go.mod h1:xBAYlnt/ay+11ShkdFKNAG7LsyK/tmNBVvVOwrfMgdI= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= @@ -147,6 +153,8 @@ k8s.io/klog/v2 v2.140.0 h1:Tf+J3AH7xnUzZyVVXhTgGhEKnFqye14aadWv7bzXdzc= k8s.io/klog/v2 v2.140.0/go.mod h1:o+/RWfJ6PwpnFn7OyAG3QnO47BFsymfEfrz6XyYSSp0= k8s.io/kube-openapi v0.0.0-20260317180543-43fb72c5454a h1:xCeOEAOoGYl2jnJoHkC3hkbPJgdATINPMAxaynU2Ovg= k8s.io/kube-openapi v0.0.0-20260317180543-43fb72c5454a/go.mod h1:uGBT7iTA6c6MvqUvSXIaYZo9ukscABYi2btjhvgKGZ0= +k8s.io/streaming v0.36.4 h1:RS5YlhrdBN2pKGVjgygGntdu6SNdsduyjGWGe3cX0vo= +k8s.io/streaming v0.36.4/go.mod h1:tJ6S2bZa2HxIBauguBbCWSCYyd93Grfz1+z3tcOvlDE= k8s.io/utils v0.0.0-20260210185600-b8788abfbbc2 h1:AZYQSJemyQB5eRxqcPky+/7EdBj0xi3g0ZcxxJ7vbWU= k8s.io/utils v0.0.0-20260210185600-b8788abfbbc2/go.mod h1:xDxuJ0whA3d0I4mf/C4ppKHxXynQ+fxnkmQH0vTHnuk= sigs.k8s.io/cli-utils v0.37.2 h1:GOfKw5RV2HDQZDJlru5KkfLO1tbxqMoyn1IYUxqBpNg= diff --git a/hack/smoke/agent.app.yaml b/hack/smoke/agent.app.yaml new file mode 100644 index 0000000..5df9d41 --- /dev/null +++ b/hack/smoke/agent.app.yaml @@ -0,0 +1,15 @@ +# The app smoke-agent deploys: two whoami components, one at / and one at /api with stripPrefix, +# so that a request shows which component got it, under which path and with which Host. +apiVersion: shelf.dev/v1alpha1 +name: agent +components: + web: + image: traefik/whoami:v1.11.0 + port: 80 + route: / + env: + WHOAMI_NAME: web-v1 + api: + image: traefik/whoami:v1.11.0 + port: 80 + route: { path: /api, stripPrefix: true } diff --git a/hack/smoke/agent.sh b/hack/smoke/agent.sh new file mode 100755 index 0000000..981aded --- /dev/null +++ b/hack/smoke/agent.sh @@ -0,0 +1,156 @@ +#!/usr/bin/env bash +# Checks the Phase 9 agent interface against the dev cluster, with the dev registry standing in +# for GHCR: `shelf app status --json`, `--wait --commit` that returns Ready only after the +# awaited commit rolled out, Failed (exit 1) for an image that cannot be pulled, TimedOut +# (exit 2) for a commit that never arrives, and `shelf app open` with the app's Host and +# stripPrefix through Traefik. +# +# Installs or updates the platform first (idempotent). Needs network access for images. +set -euo pipefail +source "$(dirname "$0")/../lib.sh" +require_devcontainer + +repo="$(cd "$(dirname "$0")/../.." && pwd)" +work="$(mktemp -d)" +app=agent-smoke +artifact="oci://$SHELF_REGISTRY_HOST/smoke/$app-deploy" +open_pid="" +export SHELF_HOME="$work/home" + +shelf() { "$work/shelf" "$@"; } + +cleanup() { + [[ -n "$open_pid" ]] && kill "$open_pid" 2>/dev/null || true + if [[ "${KEEP:-}" == 1 ]]; then + echo "KEEP=1: leaving app $app in place; its secret backup is in $SHELF_HOME" + return + fi + shelf app rm "$app" --yes >/dev/null 2>&1 || true + rm -rf "$work" +} + +step() { echo "--- $*"; } + +# commit: a random 40-digit commit hash, as the reusable workflow stamps on an artifact. +commit() { head -c 20 /dev/urandom | od -An -tx1 | tr -d ' \n'; } + +# publish : pushes the files in dir as the deploy artifact, tag main, built from +# commit. +publish() { + flux push artifact "$artifact:main" --path "$1" --source local --revision "main@sha1:$2" \ + --insecure-registry >/dev/null 2>&1 || die "push failed" +} + +# render : writes the deploy manifest of an app.yaml into dir. +render() { + mkdir -p "$2" + (cd "$repo" && go run ./cmd/shelf render "$1") >"$2/configmap.yaml" 2>/dev/null +} + +# verdict : one field of a WaitResult. +verdict() { jq -r "$2" "$1"; } + +for leftover in "namespace/$app" "-n shelf-system secret/app-$app" \ + "-n shelf-system resourcesetinputprovider/$app"; do + # shellcheck disable=SC2086 + kubectl get $leftover >/dev/null 2>&1 \ + && die "$leftover already exists; remove it first (shelf app rm $app)" +done +trap cleanup EXIT + +step "build shelf, push platform and chart, init cluster" +(cd "$repo" && go build -o "$work/shelf" ./cmd/shelf) +"$repo/hack/platform-push.sh" >/dev/null 2>&1 +"$repo/hack/chart-push.sh" >/dev/null 2>&1 +shelf init cluster --yes --host-suffix "$(cluster_host_suffix)" --insecure-registry \ + --platform "oci://$SHELF_REGISTRY_HOST/shelf/platform:dev" \ + --chart "oci://$SHELF_REGISTRY_HOST/shelf/charts/shelf-app:0.0.0-dev" >/dev/null + +step "add the app at a first commit" +c1="$(commit)" +render "$repo/hack/smoke/agent.app.yaml" "$work/v1" +publish "$work/v1" "$c1" +shelf app add "$app" "$artifact:main" --insecure-registry >/dev/null + +step "shelf app status --json" +shelf app status "$app" --json >"$work/status.json" +[[ "$(jq -r .kind "$work/status.json")" == AppStatus ]] || die "not an AppStatus: $(cat "$work/status.json")" +[[ "$(jq -r .commit "$work/status.json")" == "$c1" ]] || die "status does not name commit $c1" +[[ "$(jq '.stages | length' "$work/status.json")" == 5 ]] || die "status does not hold five stages" +shelf app status --json | jq -e --arg app "$app" '.kind == "AppList" and any(.items[]; .name == $app)' >/dev/null \ + || die "the AppList lacks $app" +shelf app status nope --json >"$work/error.json" 2>/dev/null && die "an unknown app was found" +[[ "$(jq -r .kind "$work/error.json")" == Error ]] || die "an error is not an Error document" + +step "--wait --commit waits for the commit to roll out" +c2="$(commit)" +sed 's/web-v1/web-v2/' "$repo/hack/smoke/agent.app.yaml" >"$work/v2.app.yaml" +render "$work/v2.app.yaml" "$work/v2" +start=$SECONDS +shelf app status "$app" --wait --commit "$c2" --json >"$work/wait2.json" & +wait_pid=$! +sleep 5 +kill -0 "$wait_pid" 2>/dev/null || die "the wait returned before commit $c2 was pushed: $(cat "$work/wait2.json")" +publish "$work/v2" "$c2" +code=0 +wait "$wait_pid" || code=$? +echo "wait: $((SECONDS - start)) s" +[[ "$code" == 0 && "$(verdict "$work/wait2.json" .outcome)" == Ready ]] \ + || die "wait for $c2: exit $code, $(cat "$work/wait2.json")" +[[ "$(verdict "$work/wait2.json" .commit)" == "$c2" ]] || die "Ready for another commit: $(cat "$work/wait2.json")" +kubectl -n "$app" get deploy web -o jsonpath='{.spec.template.spec.containers[0].env}' | grep -q web-v2 \ + || die "Ready, but the release does not carry the new values" +kubectl -n "$app" rollout status deploy/web --timeout=10s >/dev/null || die "Ready before the rollout finished" + +step "shelf app open reaches the app with its own Host and stripPrefix" +port="$(python3 -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1])')" +shelf app open "$app" --port "$port" >"$work/open.txt" 2>&1 & +open_pid=$! +up() { curl -fsS "http://127.0.0.1:$port/" >/dev/null 2>&1; } +for _ in $(seq 30); do up && break; sleep 1; done +up || die "shelf app open does not answer: $(cat "$work/open.txt")" +api="$(curl -fsS "http://127.0.0.1:$port/api/x")" +grep -q '^GET /x ' <<<"$api" || die "/api/x did not reach api as /x: $api" +grep -q "^Host: $app.shelf.internal" <<<"$api" || die "Traefik did not get the app's Host: $api" +grep -q '^Hostname: api-' <<<"$api" || die "/api/x was not answered by api: $api" +web="$(curl -fsS "http://127.0.0.1:$port/")" +grep -q '^Name: web-v2' <<<"$web" || die "/ was not answered by web at v2: $web" +ss -ltn | grep -q "127.0.0.1:$port " || die "not listening on 127.0.0.1:$port" +ss -ltn | grep -E "(0\.0\.0\.0|\*|\[::\]):$port " && die "listening beyond the loopback interface" +kill "$open_pid" +wait "$open_pid" 2>/dev/null || true +open_pid="" + +step "an image that cannot be pulled ends the wait with Failed" +c3="$(commit)" +render "$repo/hack/smoke/agent.app.yaml" "$work/v3" +sed -i "s/@sha256:[0-9a-f]*/@sha256:$(printf '0%.0s' $(seq 64))/" "$work/v3/configmap.yaml" +publish "$work/v3" "$c3" +code=0 +shelf app status "$app" --wait --commit "$c3" --timeout 5m --json >"$work/wait3.json" || code=$? +[[ "$code" == 1 && "$(verdict "$work/wait3.json" .outcome)" == Failed ]] \ + || die "broken image: exit $code, $(cat "$work/wait3.json")" +[[ "$(verdict "$work/wait3.json" .stage.name)" == "the pods" ]] || die "not the pods: $(cat "$work/wait3.json")" +verdict "$work/wait3.json" .stage.hint | grep -q "cannot be pulled" || die "no pull hint: $(cat "$work/wait3.json")" + +step "a commit that never arrives ends the wait with TimedOut" +code=0 +shelf app status "$app" --wait --commit deadbeef --timeout 5s --json >"$work/wait4.json" || code=$? +[[ "$code" == 2 && "$(verdict "$work/wait4.json" .outcome)" == TimedOut ]] \ + || die "unknown commit: exit $code, $(cat "$work/wait4.json")" +verdict "$work/wait4.json" .stage.message | grep -q "waiting for commit deadbee" \ + || die "the timeout does not say what it waited for: $(cat "$work/wait4.json")" + +# The agent's loop ends with a fix: a working commit after the broken one is Ready again. It +# also leaves no release stuck in an upgrade, which would hold up the removal. +step "a fixed commit after the failure is Ready again" +c4="$(commit)" +publish "$work/v2" "$c4" +code=0 +shelf app status "$app" --wait --commit "$c4" --timeout 8m --json >"$work/wait5.json" || code=$? +[[ "$code" == 0 && "$(verdict "$work/wait5.json" .outcome)" == Ready ]] \ + || die "fixed commit: exit $code, $(cat "$work/wait5.json")" + +step "remove the app" +shelf app rm "$app" --yes >/dev/null +echo "smoke-agent passed" diff --git a/internal/cli/app.go b/internal/cli/app.go index 6befde9..424865f 100644 --- a/internal/cli/app.go +++ b/internal/cli/app.go @@ -19,6 +19,7 @@ func newAppCmd(o Options) *cobra.Command { cmd.AddCommand( newAppAddCmd(o), newAppRmCmd(o), newAppStatusCmd(o), newAppRedeployCmd(o), newAppSecretsCmd(o), newAppCredentialsCmd(o), newAppTagsCmd(o), + newAppOpenCmd(o), ) return cmd } diff --git a/internal/cli/app_test.go b/internal/cli/app_test.go index 23d4671..fc05848 100644 --- a/internal/cli/app_test.go +++ b/internal/cli/app_test.go @@ -277,3 +277,28 @@ func TestAppStatusComponents(t *testing.T) { }) } } + +func TestAppStatusShowsTheCommit(t *testing.T) { + t.Parallel() + const sha = "42a964eb0b4bd7248305d5f22233d6dbabcd56f3" + for name, commit := range map[string]string{"with a commit": sha, "without one": ""} { + t.Run(name, func(t *testing.T) { + t.Parallel() + h := newHarness(t).public() + h.cluster.states = []cluster.AppState{{Name: "hello", Phase: cluster.PhaseReady, + Revision: "main@sha256:abc", Commit: commit}} + + stdout, stderr, code := h.run(t, "app", "status", "hello") + if code != 0 { + t.Fatalf("exit code %d: %s", code, stderr) + } + line := " commit " + sha + if got := strings.Contains(stdout, line); got != (commit != "") { + t.Errorf("commit line shown %v, want %v:\n%s", got, commit != "", stdout) + } + if commit == "" && strings.Contains(stdout, "commit") { + t.Errorf("an artifact without a commit shows a commit line:\n%s", stdout) + } + }) + } +} diff --git a/internal/cli/choices.go b/internal/cli/choices.go index a333f5f..16af0db 100644 --- a/internal/cli/choices.go +++ b/internal/cli/choices.go @@ -78,7 +78,10 @@ func newConnectionZonesCmd(o Options) *cobra.Command { } func newAppTagsCmd(o Options) *cobra.Command { - var target clusterFlags + var ( + target clusterFlags + asJSON bool + ) cmd := &cobra.Command{ Use: "tags ", Short: "List the tags of an app's deploy artifact, newest first", @@ -103,6 +106,9 @@ one with ` + "`shelf app add :`" + `.`, return err } out := cmd.OutOrStdout() + if asJSON { + return printJSON(out, tagList(app.Name, app.Artifact.Tag, tags)) + } if len(tags) == 0 { fmt.Fprintln(out, "No tags.") return nil @@ -119,6 +125,7 @@ one with ` + "`shelf app add :`" + `.`, }, } target.registerTarget(cmd.Flags()) + registerJSON(cmd, &asJSON) return cmd } diff --git a/internal/cli/cli.go b/internal/cli/cli.go index 37ea2f5..963f9ac 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -8,7 +8,9 @@ import ( "io" "maps" "os" + "os/exec" "path/filepath" + "runtime" "runtime/debug" "slices" "strings" @@ -44,6 +46,20 @@ type Options struct { // NewOps builds the operations against a cluster. Nil means ops.New, which talks to the // real cluster, registry and Cloudflare. NewOps func(ops.Target, ops.Env) *ops.Ops + // OpenBrowser opens an address in the default browser. Nil means `open` on macOS and + // `xdg-open` elsewhere. + OpenBrowser func(url string) error +} + +func (o Options) openBrowser(url string) error { + if o.OpenBrowser != nil { + return o.OpenBrowser(url) + } + name := "xdg-open" + if runtime.GOOS == "darwin" { + name = "open" + } + return exec.Command(name, url).Start() } func (o Options) getenv(key string) string { @@ -296,14 +312,22 @@ func newSchemaCmd() *cobra.Command { } func newVersionCmd(o Options) *cobra.Command { - return &cobra.Command{ + var asJSON bool + cmd := &cobra.Command{ Use: "version", Short: "Print the shelf version", Args: cobra.NoArgs, - Run: func(cmd *cobra.Command, _ []string) { + RunE: func(cmd *cobra.Command, _ []string) error { + if asJSON { + return printJSON(cmd.OutOrStdout(), versionDocument{header: newHeader("Version"), + Version: o.version(), AppAPIVersion: schema.APIVersion}) + } fmt.Fprintf(cmd.OutOrStdout(), "shelf %s (app.yaml %s)\n", o.version(), schema.APIVersion) + return nil }, } + registerJSON(cmd, &asJSON) + return cmd } // buildVersion is the version stamped into this binary, or a marker that it is a development @@ -323,12 +347,28 @@ func buildVersion() string { } // Execute runs the command line and returns the process exit code. +// ExitError ends a command with a status other than 1, after it has printed what that status +// means; Execute prints nothing more. +type ExitError struct{ Code int } + +func (e *ExitError) Error() string { return fmt.Sprintf("exit status %d", e.Code) } + func Execute(ctx context.Context, cmd *cobra.Command) int { - if err := cmd.ExecuteContext(ctx); err != nil { - if !errors.Is(err, ErrReported) { - fmt.Fprintf(cmd.ErrOrStderr(), "error: %v\n", err) - } + ran, err := cmd.ExecuteContextC(ctx) + if err == nil { + return 0 + } + if errors.Is(err, ErrReported) { return 1 } - return 0 + if exit := (*ExitError)(nil); errors.As(err, &exit) { + return exit.Code + } + // Run with --json, the error is a document on stdout as well, so that a script reads one + // place whatever happened, and a person watching still sees it on stderr. + if ran != nil && wantsJSON(ran) { + _ = printJSON(ran.OutOrStdout(), errorDocument{header: newHeader("Error"), Message: err.Error()}) + } + fmt.Fprintf(cmd.ErrOrStderr(), "error: %v\n", err) + return 1 } diff --git a/internal/cli/connection.go b/internal/cli/connection.go index f8ee335..91df603 100644 --- a/internal/cli/connection.go +++ b/internal/cli/connection.go @@ -95,7 +95,10 @@ it gets the new token; it cannot move to another account while apps are exposed } func newConnectionListCmd(o Options) *cobra.Command { - var target clusterFlags + var ( + target clusterFlags + asJSON bool + ) cmd := &cobra.Command{ Use: "list", Short: "List the connections and the apps that use them", @@ -110,6 +113,9 @@ func newConnectionListCmd(o Options) *cobra.Command { return err } out := cmd.OutOrStdout() + if asJSON { + return printJSON(out, connectionList(conns)) + } if len(conns.Registry)+len(conns.Cloudflare) == 0 { fmt.Fprintln(out, "No connections yet.") return nil @@ -129,6 +135,7 @@ func newConnectionListCmd(o Options) *cobra.Command { }, } target.registerTarget(cmd.Flags()) + registerJSON(cmd, &asJSON) return cmd } diff --git a/internal/cli/harness_test.go b/internal/cli/harness_test.go index 0280bc1..629c549 100644 --- a/internal/cli/harness_test.go +++ b/internal/cli/harness_test.go @@ -56,7 +56,12 @@ type fakeCluster struct { states []cluster.AppState components []cluster.Component stages []cluster.Stage - found bool + // rollouts are what AppRollout returns, one per call; the last one is repeated. + rollouts []cluster.Rollout + rolloutCalls int + // traefik is the address ForwardTraefik hands out, empty when no Traefik pod is ready. + traefik string + found bool // noPlatform is a cluster that init cluster never installed the platform into, oldPlatform // one whose platform predates quick tunnels. noPlatform bool @@ -145,6 +150,22 @@ func (f *fakeCluster) AppDiagnosis(_ context.Context, _ *rest.Config, app string return cluster.Diagnosis{Name: app, Stages: f.stages}, nil } +func (f *fakeCluster) AppRollout(_ context.Context, _ *rest.Config, app string) (cluster.Rollout, error) { + if !slices.ContainsFunc(f.states, func(s cluster.AppState) bool { return s.Name == app }) { + return cluster.Rollout{}, errors.New("app " + app + " does not exist") + } + r := f.rollouts[min(f.rolloutCalls, len(f.rollouts)-1)] + f.rolloutCalls++ + return r, nil +} + +func (f *fakeCluster) ForwardTraefik(context.Context, *rest.Config) (*cluster.Forward, error) { + if f.traefik == "" { + return nil, errors.New("no Traefik pod is ready") + } + return &cluster.Forward{Addr: f.traefik, Done: make(chan struct{})}, nil +} + func (f *fakeCluster) Settings(context.Context, *rest.Config) (cluster.Settings, error) { return f.settings, f.settingsErr } @@ -208,6 +229,8 @@ type harness struct { fetched string insecure bool pull authn.Authenticator + // browser is what --browser opens an address with. + browser func(url string) error } func newHarness(t *testing.T) *harness { @@ -247,6 +270,12 @@ func (h *harness) options() Options { Images: images, Version: h.version, Getenv: func(key string) string { return h.env[key] }, + OpenBrowser: func(url string) error { + if h.browser == nil { + return errors.New("no browser in tests") + } + return h.browser(url) + }, NewOps: func(target ops.Target, env ops.Env) *ops.Ops { return &ops.Ops{ Target: target, @@ -270,6 +299,12 @@ func (h *harness) run(t *testing.T, args ...string) (stdout, stderr string, code } func (h *harness) runWithInput(t *testing.T, stdin string, args ...string) (stdout, stderr string, code int) { + t.Helper() + return h.runContext(t, context.Background(), stdin, args...) +} + +// runContext runs a command until it ends or ctx is cancelled, as Ctrl-C does. +func (h *harness) runContext(t *testing.T, ctx context.Context, stdin string, args ...string) (stdout, stderr string, code int) { t.Helper() cmd := New(h.options()) var out, errOut bytes.Buffer @@ -277,7 +312,7 @@ func (h *harness) runWithInput(t *testing.T, stdin string, args ...string) (stdo cmd.SetErr(&errOut) cmd.SetIn(strings.NewReader(stdin)) cmd.SetArgs(append(args, "--kubeconfig", h.kubeconfig)) - code = Execute(context.Background(), cmd) + code = Execute(ctx, cmd) return out.String(), errOut.String(), code } diff --git a/internal/cli/json.go b/internal/cli/json.go new file mode 100644 index 0000000..e24885f --- /dev/null +++ b/internal/cli/json.go @@ -0,0 +1,267 @@ +package cli + +import ( + "encoding/json" + "io" + "time" + + "github.com/spf13/cobra" + + "github.com/tweinmann/shelf/internal/cluster" + "github.com/tweinmann/shelf/internal/ops" +) + +// The documents `--json` prints are the contract with the agents that read them, so they are +// types of their own rather than the structs of ops and cluster: an internal rename must not +// reach an agent. Within one apiVersion a field is never removed, renamed or given another +// meaning; new fields may be added. Empty fields are left out, never null. Every kind has a +// golden file in testdata/json. + +// documentAPIVersion is the version of every document --json prints. +const documentAPIVersion = "shelf.dev/v1alpha1" + +// jsonFlag is the name of the flag, looked up again by Execute to report an error as a document. +const jsonFlag = "json" + +// header starts every document. +type header struct { + APIVersion string `json:"apiVersion"` + Kind string `json:"kind"` +} + +func newHeader(kind string) header { return header{APIVersion: documentAPIVersion, Kind: kind} } + +// registerJSON adds --json to a command. +func registerJSON(cmd *cobra.Command, value *bool) { + cmd.Flags().BoolVar(value, jsonFlag, false, + "print one JSON document ("+documentAPIVersion+") instead of text, for scripts and agents") +} + +// wantsJSON reports whether a command was run with --json. +func wantsJSON(cmd *cobra.Command) bool { + flag := cmd.Flags().Lookup(jsonFlag) + return flag != nil && flag.Value.String() == "true" +} + +func printJSON(w io.Writer, doc any) error { + enc := json.NewEncoder(w) + enc.SetIndent("", " ") + return enc.Encode(doc) +} + +// errorDocument is what a command run with --json prints when it fails. +type errorDocument struct { + header + Message string `json:"message"` +} + +// appSummary is one app in an AppList. +type appSummary struct { + Name string `json:"name"` + Phase cluster.Phase `json:"phase"` + Stage string `json:"stage,omitempty"` + Reason string `json:"reason,omitempty"` + Artifact string `json:"artifact"` + Revision string `json:"revision,omitempty"` + Commit string `json:"commit,omitempty"` + Host string `json:"host,omitempty"` + URL string `json:"url,omitempty"` + Deployed *time.Time `json:"deployed,omitempty"` +} + +type appListDocument struct { + header + Items []appSummary `json:"items"` +} + +// exposure says how an app reaches the internet, if it does. +type exposure struct { + // Kind is none, cloudflare or quick. + Kind string `json:"kind"` + Connection string `json:"connection,omitempty"` + Account string `json:"account,omitempty"` + // ConnectionMissing means this machine does not hold the app's Cloudflare connection. + ConnectionMissing bool `json:"connectionMissing,omitempty"` + Domain string `json:"domain,omitempty"` +} + +type componentDocument struct { + Name string `json:"name"` + Phase cluster.Phase `json:"phase"` + Path string `json:"path,omitempty"` + Address string `json:"address,omitempty"` + URL string `json:"url,omitempty"` + Internal bool `json:"internal,omitempty"` +} + +type stageDocument struct { + Name string `json:"name"` + Object string `json:"object"` + Phase cluster.Phase `json:"phase"` + Reason string `json:"reason,omitempty"` + Message string `json:"message,omitempty"` + Hint string `json:"hint,omitempty"` +} + +type appStatusDocument struct { + header + Name string `json:"name"` + Phase cluster.Phase `json:"phase"` + Host string `json:"host,omitempty"` + URL string `json:"url,omitempty"` + Artifact string `json:"artifact"` + Insecure bool `json:"insecure,omitempty"` + Registry string `json:"registry,omitempty"` + Exposure exposure `json:"exposure"` + Revision string `json:"revision,omitempty"` + Commit string `json:"commit,omitempty"` + Deployed *time.Time `json:"deployed,omitempty"` + Components []componentDocument `json:"components"` + Stages []stageDocument `json:"stages"` +} + +type waitResultDocument struct { + header + // Outcome is Ready, Failed or TimedOut. + Outcome ops.Outcome `json:"outcome"` + App string `json:"app"` + Commit string `json:"commit,omitempty"` + Revision string `json:"revision,omitempty"` + // Stage is the first stage that is not Ready; it is left out when the outcome is Ready. + Stage *stageDocument `json:"stage,omitempty"` +} + +func waitResult(r ops.WaitResult) waitResultDocument { + doc := waitResultDocument{header: newHeader("WaitResult"), Outcome: r.Outcome, App: r.App, + Commit: r.Commit, Revision: r.Revision} + if r.Stage != nil { + s := stage(*r.Stage) + doc.Stage = &s + } + return doc +} + +type tagDocument struct { + Name string `json:"name"` + Created *time.Time `json:"created,omitempty"` + Current bool `json:"current,omitempty"` +} + +type tagListDocument struct { + header + App string `json:"app"` + Items []tagDocument `json:"items"` +} + +type connectionDocument struct { + // Kind is registry or cloudflare. + Kind string `json:"kind"` + Name string `json:"name"` + // User and Registry belong to a registry connection. + User string `json:"user,omitempty"` + Registry string `json:"registry,omitempty"` + // Account belongs to a Cloudflare connection; Missing means this machine does not hold it. + Account string `json:"account,omitempty"` + Missing bool `json:"missing,omitempty"` + Apps []string `json:"apps"` +} + +type connectionListDocument struct { + header + Items []connectionDocument `json:"items"` +} + +type versionDocument struct { + header + Version string `json:"version"` + // AppAPIVersion is the apiVersion of the app.yaml files this binary reads. + AppAPIVersion string `json:"appAPIVersion"` +} + +// timestamp is a time in UTC, or nil for the zero time, so that the field is left out. +func timestamp(t time.Time) *time.Time { + if t.IsZero() { + return nil + } + t = t.UTC() + return &t +} + +func appList(apps []ops.App) appListDocument { + doc := appListDocument{header: newHeader("AppList"), Items: []appSummary{}} + for _, a := range apps { + doc.Items = append(doc.Items, appSummary{ + Name: a.Name, Phase: a.Phase, Stage: a.Stage, Reason: a.Reason, + Artifact: a.Artifact.String(), Revision: a.Revision, Commit: a.Commit, + Host: a.Host, URL: a.URL, Deployed: timestamp(a.Deployed), + }) + } + return doc +} + +func appStatus(a ops.App, d cluster.Diagnosis) appStatusDocument { + doc := appStatusDocument{ + header: newHeader("AppStatus"), + Name: a.Name, Phase: a.Phase, Host: a.Host, URL: a.URL, + Artifact: a.Artifact.String(), Insecure: a.Insecure, Registry: a.Registry, + Exposure: exposure{Kind: "none"}, + Revision: a.Revision, Commit: a.Commit, Deployed: timestamp(a.Deployed), + Components: []componentDocument{}, + Stages: stages(d), + } + switch { + case a.Cloudflare != "": + doc.Exposure = exposure{Kind: "cloudflare", Connection: a.Cloudflare, Account: a.CloudflareAccount, + ConnectionMissing: a.CloudflareMissing, Domain: a.Domain} + case a.Quick: + doc.Exposure = exposure{Kind: "quick"} + } + for _, c := range a.Components { + doc.Components = append(doc.Components, componentDocument{ + Name: c.Name, Phase: c.Phase, Path: c.Path, Address: c.Address, URL: c.URL, Internal: c.Internal, + }) + } + return doc +} + +func stages(d cluster.Diagnosis) []stageDocument { + out := make([]stageDocument, 0, len(d.Stages)) + for _, s := range d.Stages { + out = append(out, stage(s)) + } + return out +} + +func stage(s cluster.Stage) stageDocument { + return stageDocument{Name: s.Name, Object: s.Object, Phase: s.Phase, Reason: s.Reason, + Message: s.Message, Hint: s.Hint} +} + +func tagList(app string, current string, tags []ops.Tag) tagListDocument { + doc := tagListDocument{header: newHeader("TagList"), App: app, Items: []tagDocument{}} + for _, t := range tags { + doc.Items = append(doc.Items, tagDocument{Name: t.Name, Created: timestamp(t.Created), Current: t.Name == current}) + } + return doc +} + +func connectionList(conns ops.Connections) connectionListDocument { + doc := connectionListDocument{header: newHeader("ConnectionList"), Items: []connectionDocument{}} + for _, c := range conns.Registry { + doc.Items = append(doc.Items, connectionDocument{Kind: "registry", Name: c.Name, User: c.Username, + Registry: cluster.RegistryHost, Apps: orEmpty(c.Apps)}) + } + for _, c := range conns.Cloudflare { + doc.Items = append(doc.Items, connectionDocument{Kind: "cloudflare", Name: c.Name, Account: c.Account, + Missing: c.Missing, Apps: orEmpty(c.Apps)}) + } + return doc +} + +// orEmpty keeps a list a list in JSON: an unused connection has "apps": [], not null. +func orEmpty(s []string) []string { + if s == nil { + return []string{} + } + return s +} diff --git a/internal/cli/json_test.go b/internal/cli/json_test.go new file mode 100644 index 0000000..ffc6f26 --- /dev/null +++ b/internal/cli/json_test.go @@ -0,0 +1,183 @@ +package cli + +import ( + "bytes" + "context" + "encoding/json" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/tweinmann/shelf/internal/cluster" + "github.com/tweinmann/shelf/internal/testutil" +) + +const testCommit = "42a964eb0b4bd7248305d5f22233d6dbabcd56f3" + +// fiveStages is the chain of a healthy app; failing replaces the pods stage. +func fiveStages(pods cluster.Stage) []cluster.Stage { + return []cluster.Stage{ + {Name: "the deploy artifact", Object: "OCIRepository hello/deploy", Phase: cluster.PhaseReady, + Reason: "Succeeded", Message: "stored artifact for digest 'main@sha256:abc'"}, + {Name: "the app values", Object: "Kustomization hello/deploy", Phase: cluster.PhaseReady, + Reason: "ReconciliationSucceeded", Message: "Applied revision: main@sha256:abc"}, + {Name: "the chart", Object: "OCIRepository hello/shelf-app", Phase: cluster.PhaseReady, + Reason: "Succeeded", Message: "stored artifact for digest '0.6.0@sha256:def'"}, + {Name: "the app", Object: "HelmRelease hello/hello", Phase: cluster.PhaseReady, + Reason: "UpgradeSucceeded", Message: "Helm upgrade succeeded"}, + pods, + } +} + +var ( + podsReady = cluster.Stage{Name: "the pods", Object: "Pods", Phase: cluster.PhaseReady, Message: "2 running"} + podsFailed = cluster.Stage{Name: "the pods", Object: "Pod hello/web-6d4f", Phase: cluster.PhaseFailed, + Reason: "ImagePullBackOff", Message: "Back-off pulling image", + Hint: "the image cannot be pulled; check the reference and whether the cluster may read that registry"} +) + +func helloState(phase cluster.Phase) cluster.AppState { + return cluster.AppState{ + Name: "hello", Artifact: cluster.Artifact{URL: "oci://ghcr.io/o/hello", Tag: "main"}, + Registry: "ghcr", Revision: "main@sha256:abc", Commit: testCommit, + Deployed: time.Date(2026, 10, 27, 14, 3, 0, 0, time.FixedZone("CET", 3600)), + Phase: phase, + } +} + +// runJSON runs a command with --json and checks what every document has to be: one object on +// stdout, with the envelope, and no null anywhere. +func runJSON(t *testing.T, h *harness, wantCode int, args ...string) []byte { + t.Helper() + stdout, stderr, code := h.run(t, append(args, "--json")...) + if code != wantCode { + t.Fatalf("%v: exit code %d, want %d: %s", args, code, wantCode, stderr) + } + var doc map[string]any + if err := json.Unmarshal([]byte(stdout), &doc); err != nil { + t.Fatalf("%v: stdout is not one JSON object: %v\n%s", args, err, stdout) + } + if doc["apiVersion"] != documentAPIVersion || doc["kind"] == "" { + t.Errorf("%v: envelope %v %v", args, doc["apiVersion"], doc["kind"]) + } + if strings.Contains(stdout, "null") { + t.Errorf("%v: a field is null:\n%s", args, stdout) + } + for _, token := range []string{"cf-secret-token", "ghcr-secret-token"} { + if strings.Contains(stdout+stderr, token) { + t.Fatalf("%v: a token appears in the output", args) + } + } + return []byte(stdout) +} + +func goldenJSON(t *testing.T, name string, got []byte) { + t.Helper() + testutil.Golden(t, filepath.Join("testdata", "json", name+".json"), got) +} + +func TestJSONAppList(t *testing.T) { + t.Parallel() + h := newHarness(t).public() + goldenJSON(t, "applist-empty", runJSON(t, h, 0, "app", "status")) + + failing := helloState(cluster.PhaseFailed) + failing.Name, failing.Commit, failing.Deployed = "greeter", "", time.Time{} + failing.Stage, failing.Reason = "the pods", "Back-off pulling image" + h.cluster.states = []cluster.AppState{failing, helloState(cluster.PhaseReady)} + goldenJSON(t, "applist", runJSON(t, h, 0, "app", "status")) +} + +func TestJSONAppStatus(t *testing.T) { + t.Parallel() + components := []cluster.Component{ + {Name: "db", Ports: []cluster.Port{{Name: "main", Number: 5432}}, Phase: cluster.PhaseReady}, + {Name: "web", Ports: []cluster.Port{{Name: "main", Number: 80}}, Path: "/", Phase: cluster.PhaseReady}, + } + tests := map[string]struct { + phase cluster.Phase + pods cluster.Stage + }{ + "appstatus": {cluster.PhaseReady, podsReady}, + "appstatus-failing": {cluster.PhaseFailed, podsFailed}, + } + for name, tt := range tests { + t.Run(name, func(t *testing.T) { + t.Parallel() + h := newHarness(t).public() + h.cluster.states = []cluster.AppState{helloState(tt.phase)} + h.cluster.components, h.cluster.stages = components, fiveStages(tt.pods) + goldenJSON(t, name, runJSON(t, h, 0, "app", "status", "hello")) + }) + } +} + +func TestJSONError(t *testing.T) { + t.Parallel() + h := newHarness(t).public() + stdout, stderr, code := h.run(t, "app", "status", "nope", "--json") + want := `{"apiVersion":"shelf.dev/v1alpha1","kind":"Error","message":"app nope does not exist"}` + var got, wantDoc any + _ = json.Unmarshal([]byte(want), &wantDoc) + if err := json.Unmarshal([]byte(stdout), &got); err != nil || !jsonEqual(got, wantDoc) { + t.Errorf("stdout %s, want %s", stdout, want) + } + if code != 1 || !strings.Contains(stderr, "app nope does not exist") { + t.Errorf("exit code %d, stderr %q", code, stderr) + } + + // Without --json, an error stays text on stderr only. + stdout, _, _ = h.run(t, "app", "status", "nope") + if stdout != "" { + t.Errorf("text mode printed %q on stdout", stdout) + } +} + +func jsonEqual(a, b any) bool { + x, _ := json.Marshal(a) + y, _ := json.Marshal(b) + return string(x) == string(y) +} + +func TestJSONTagList(t *testing.T) { + t.Parallel() + h := newHarness(t) + h.app = appWithSecrets("greeter") + h.defineRegistry(t, "ghcr", "o", "ghcr-secret-token") + h.mustRun(t, "app", "add", "greeter", greeterArtifact, "--registry", "ghcr") + goldenJSON(t, "taglist", runJSON(t, h, 0, "app", "tags", "greeter")) +} + +func TestJSONConnectionList(t *testing.T) { + t.Parallel() + h := cloudflareHarness(t) + goldenJSON(t, "connectionlist-unused", runJSON(t, h, 0, "connection", "list")) + + h.defineRegistry(t, "ghcr", "o", "ghcr-secret-token") + h.mustRun(t, "app", "add", "greeter", greeterArtifact, "--registry", "ghcr", + "--cloudflare", "home", "--domain", "example.com") + goldenJSON(t, "connectionlist", runJSON(t, h, 0, "connection", "list")) +} + +func TestJSONVersion(t *testing.T) { + t.Parallel() + // version takes no cluster flags, so it runs without the harness's --kubeconfig. + cmd := New(newHarness(t).options()) + var out bytes.Buffer + cmd.SetOut(&out) + cmd.SetArgs([]string{"version", "--json"}) + if code := Execute(context.Background(), cmd); code != 0 { + t.Fatalf("exit code %d", code) + } + goldenJSON(t, "version", out.Bytes()) +} + +// TestNoJSONForSecrets keeps secret values out of anything an agent would parse and keep. +func TestNoJSONForSecrets(t *testing.T) { + t.Parallel() + _, stderr, code := newHarness(t).run(t, "app", "secrets", "hello", "--json") + if code == 0 || !strings.Contains(stderr, "unknown flag: --json") { + t.Errorf("exit code %d, stderr %q", code, stderr) + } +} diff --git a/internal/cli/open.go b/internal/cli/open.go new file mode 100644 index 0000000..6e6fd52 --- /dev/null +++ b/internal/cli/open.go @@ -0,0 +1,93 @@ +package cli + +import ( + "context" + "errors" + "fmt" + "net" + "net/http" + "strconv" + "time" + + "github.com/spf13/cobra" + + "github.com/tweinmann/shelf/internal/ops" +) + +// openAddress is the only address `shelf app open` listens on. There is no flag for another one: +// a listener on the LAN would be an exposure, and exposure is the operator's decision, made +// with `shelf app add` and audit-logged, not a side effect of looking at an app. +const openAddress = "127.0.0.1" + +func newAppOpenCmd(o Options) *cobra.Command { + var ( + target clusterFlags + port int + browser bool + ) + cmd := &cobra.Command{ + Use: "open ", + Short: "Reach an app from a browser on this machine, through the cluster's Traefik", + Long: `Listen on a port of 127.0.0.1 and carry every request through the Kubernetes API server to +the cluster's Traefik, with the Host header of the app's own host. Traefik routes it exactly as it +routes a tunnel's requests, so the app's routes and stripPrefix behave as they do on the +internet. This works for apps that stay inside the cluster, too. + +This is not an exposure: nothing about the app changes, nothing listens anywhere but on this +machine's loopback interface, and the address works only for as long as this command runs. Stop +it with Ctrl-C.`, + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + name := args[0] + if err := ops.CheckAppName(name); err != nil { + return err + } + if port < 0 || port > 65535 { + return fmt.Errorf("--port %d is not a port", port) + } + shelf, err := target.load(o) + if err != nil { + return err + } + ctx := cmd.Context() + proxy, err := shelf.OpenApp(ctx, name) + if err != nil { + return err + } + defer proxy.Close() + ln, err := net.Listen("tcp", net.JoinHostPort(openAddress, strconv.Itoa(port))) + if err != nil { + return fmt.Errorf("cannot listen on %s:%d: %w", openAddress, port, err) + } + local := ln.Addr().String() + url := "http://" + local + "/" + out := cmd.OutOrStdout() + fmt.Fprintf(out, "%s (%s) is at %s\n", name, proxy.Host, url) + fmt.Fprintln(cmd.ErrOrStderr(), "Only this machine can reach it, and only while this runs. Press Ctrl-C to stop.") + + srv := &http.Server{Handler: proxy.Handler(local), ReadHeaderTimeout: 30 * time.Second} + served := make(chan error, 1) + go func() { served <- srv.Serve(ln) }() + if browser { + if err := o.openBrowser(url); err != nil { + fmt.Fprintf(cmd.ErrOrStderr(), "cannot open a browser: %v\n", err) + } + } + select { + case err := <-served: + return err + case <-ctx.Done(): + } + shutdown, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second) + defer cancel() + if err := srv.Shutdown(shutdown); err != nil && !errors.Is(err, http.ErrServerClosed) { + return err + } + return nil + }, + } + target.registerTarget(cmd.Flags()) + cmd.Flags().IntVar(&port, "port", 0, "the local port; a free one when not given") + cmd.Flags().BoolVar(&browser, "browser", false, "open the address in the default browser") + return cmd +} diff --git a/internal/cli/open_test.go b/internal/cli/open_test.go new file mode 100644 index 0000000..c5ed4f2 --- /dev/null +++ b/internal/cli/open_test.go @@ -0,0 +1,124 @@ +package cli + +import ( + "context" + "fmt" + "io" + "net" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" + + "github.com/tweinmann/shelf/internal/cluster" +) + +// openHarness is a cluster with the app hello and a Traefik that reports the Host it got. +func openHarness(t *testing.T) *harness { + t.Helper() + traefik := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + fmt.Fprintf(w, "host=%s path=%s", r.Host, r.URL.Path) + })) + t.Cleanup(traefik.Close) + h := newHarness(t).public() + quick := helloState(cluster.PhaseReady) + quick.Quick, quick.QuickURL = true, "https://random-words.trycloudflare.com" + h.cluster.states = []cluster.AppState{quick} + h.cluster.traefik = traefik.Listener.Addr().String() + return h +} + +func TestAppOpen(t *testing.T) { + t.Parallel() + h := openHarness(t) + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + var opened, body string + h.browser = func(url string) error { + // The browser is opened once the address answers; what it sees is checked, then the + // command is stopped as Ctrl-C stops it. + defer cancel() + opened = url + resp, err := http.Get(url + "api/x") + if err != nil { + return err + } + defer resp.Body.Close() + b, _ := io.ReadAll(resp.Body) + body = string(b) + return nil + } + stdout, stderr, code := h.runContext(t, ctx, "", "app", "open", "hello", "--browser") + if code != 0 { + t.Fatalf("exit code %d: %s", code, stderr) + } + if !strings.HasPrefix(opened, "http://127.0.0.1:") || !strings.Contains(stdout, "hello (hello.shelf.internal) is at "+opened) { + t.Errorf("opened %q, stdout %q", opened, stdout) + } + // The app's own host, not the quick tunnel's name: that is what its Ingress answers to. + if body != "host=hello.shelf.internal path=/api/x" { + t.Errorf("Traefik got %q", body) + } + if len(h.cluster.added)+len(h.cluster.removed)+len(h.cluster.redeployed) != 0 { + t.Errorf("opening changed the app") + } +} + +func TestAppOpenWithoutBrowser(t *testing.T) { + t.Parallel() + h := openHarness(t) + called := false + h.browser = func(string) error { called = true; return nil } + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + stdout, stderr, code := h.runContext(t, ctx, "", "app", "open", "hello") + if code != 0 || called { + t.Errorf("exit code %d, browser started %v: %s", code, called, stderr) + } + if !strings.Contains(stdout, "is at http://127.0.0.1:") { + t.Errorf("stdout %q", stdout) + } +} + +func TestAppOpenPort(t *testing.T) { + t.Parallel() + busy, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + defer busy.Close() + port := busy.Addr().(*net.TCPAddr).Port + _, stderr, code := openHarness(t).run(t, "app", "open", "hello", "--port", fmt.Sprint(port)) + if code != 1 || !strings.Contains(stderr, fmt.Sprintf("cannot listen on 127.0.0.1:%d", port)) { + t.Errorf("port in use: exit code %d, stderr %q", code, stderr) + } + + h := openHarness(t) + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + free, _ := net.Listen("tcp", "127.0.0.1:0") + port = free.Addr().(*net.TCPAddr).Port + free.Close() + stdout, _, _ := h.runContext(t, ctx, "", "app", "open", "hello", "--port", fmt.Sprint(port)) + if !strings.Contains(stdout, fmt.Sprintf("http://127.0.0.1:%d/", port)) { + t.Errorf("chosen port: %q", stdout) + } +} + +func TestAppOpenRefusals(t *testing.T) { + t.Parallel() + _, stderr, code := openHarness(t).run(t, "app", "open", "nope") + if code != 1 || !strings.Contains(stderr, "app nope does not exist") { + t.Errorf("unknown app: exit code %d, stderr %q", code, stderr) + } + h := openHarness(t) + h.cluster.traefik = "" + if _, stderr, code := h.run(t, "app", "open", "hello"); code != 1 || !strings.Contains(stderr, "no Traefik pod is ready") { + t.Errorf("no Traefik: exit code %d, stderr %q", code, stderr) + } + if _, stderr, code := openHarness(t).run(t, "app", "open", "hello", "--address", "0.0.0.0"); code != 1 || + !strings.Contains(stderr, "unknown flag") { + t.Errorf("another address: exit code %d, stderr %q", code, stderr) + } +} diff --git a/internal/cli/status.go b/internal/cli/status.go index 4fe7880..a92a267 100644 --- a/internal/cli/status.go +++ b/internal/cli/status.go @@ -1,6 +1,7 @@ package cli import ( + "errors" "fmt" "io" "maps" @@ -15,28 +16,67 @@ import ( ) func newAppStatusCmd(o Options) *cobra.Command { - var target clusterFlags + var ( + target clusterFlags + asJSON bool + wait bool + commit string + timeout time.Duration + ) cmd := &cobra.Command{ Use: "status [name]", Short: "Show how the apps of this cluster are doing", Long: `Without a name, list every app with its state. With a name, walk the chain from the deploy artifact to the running pods and say which step is not ready. +With --wait, wait until the app is Ready or a step failed, and print the verdict. With --commit +as well, wait first until the app runs the deploy artifact built from that git commit, so that +the verdict is about it and not about the version before. The exit status is 0 when the app is +Ready, 1 when a step failed and 2 when the time ran out. Waiting only reads the cluster; the +platform notices a new artifact within a minute by itself. + This is what the admin UI shows on its pages, in text.`, - Args: cobra.MaximumNArgs(1), + Example: ` shelf app status hello --wait --commit "$(git rev-parse HEAD)" --json`, + Args: cobra.MaximumNArgs(1), RunE: func(cmd *cobra.Command, args []string) error { + if commit != "" && !wait { + return errors.New("--commit needs --wait") + } + if wait && len(args) == 0 { + return errors.New("--wait needs the name of an app") + } + if commit != "" { + if err := ops.CheckCommit(commit); err != nil { + return err + } + } shelf, err := target.load(o) if err != nil { return err } out := cmd.OutOrStdout() if len(args) == 0 { - return listApps(cmd, shelf, out) + return listApps(cmd, shelf, out, asJSON) } name := args[0] if err := ops.CheckAppName(name); err != nil { return err } + if wait { + result, err := shelf.Wait(cmd.Context(), name, ops.WaitOptions{Commit: commit, Timeout: timeout}) + if err != nil { + return err + } + if asJSON { + err = printJSON(out, waitResult(result)) + } else { + printVerdict(out, result) + } + if err != nil { + return err + } + return verdictExit(result.Outcome) + } app, err := shelf.App(cmd.Context(), name) if err != nil { return err @@ -45,6 +85,9 @@ This is what the admin UI shows on its pages, in text.`, if err != nil { return err } + if asJSON { + return printJSON(out, appStatus(app, diagnosis)) + } printApp(out, app) printComponents(out, app.Components, app.Public || app.Quick) printChain(out, diagnosis) @@ -52,14 +95,56 @@ This is what the admin UI shows on its pages, in text.`, }, } target.registerTarget(cmd.Flags()) + registerJSON(cmd, &asJSON) + cmd.Flags().BoolVar(&wait, "wait", false, "wait until the app is Ready or a step failed, and print the verdict") + cmd.Flags().StringVar(&commit, "commit", "", "with --wait: the git commit the app has to run, at least 7 hex digits") + cmd.Flags().DurationVar(&timeout, "timeout", ops.DefaultWaitTimeout, "with --wait: how long to wait") return cmd } -func listApps(cmd *cobra.Command, shelf *ops.Ops, out io.Writer) error { +// printVerdict is the wait's result in three lines at most: the outcome, the step that is not +// Ready with its reason, and shelf's hint. +func printVerdict(out io.Writer, r ops.WaitResult) { + line := fmt.Sprintf("%s: %s", r.App, r.Outcome) + if r.Commit != "" { + line += ", running commit " + r.Commit[:min(len(r.Commit), 12)] + } + fmt.Fprintln(out, line) + if r.Stage == nil { + return + } + detail := r.Stage.Name + for _, part := range []string{r.Stage.Reason, firstLine(r.Stage.Message)} { + if part != "" { + detail += ": " + part + } + } + fmt.Fprintf(out, " %s\n", detail) + if r.Stage.Hint != "" { + fmt.Fprintf(out, " %s\n", r.Stage.Hint) + } +} + +// verdictExit turns an outcome into the command's exit status: 0, 1 or 2. +func verdictExit(outcome ops.Outcome) error { + switch outcome { + case ops.OutcomeReady: + return nil + case ops.OutcomeFailed: + return &ExitError{Code: 1} + default: + return &ExitError{Code: 2} + } +} + +func listApps(cmd *cobra.Command, shelf *ops.Ops, out io.Writer, asJSON bool) error { apps, err := shelf.Apps(cmd.Context()) if err != nil { return err } + if asJSON { + return printJSON(out, appList(apps)) + } if len(apps) == 0 { fmt.Fprintln(out, "No apps yet.") return nil @@ -99,6 +184,9 @@ func printApp(out io.Writer, app ops.App) { if app.Revision != "" { fmt.Fprintf(out, " running %s\n", app.Revision) } + if app.Commit != "" { + fmt.Fprintf(out, " commit %s\n", app.Commit) + } if !app.Deployed.IsZero() { fmt.Fprintf(out, " deployed %s\n", app.Deployed.Local().Format(time.RFC3339)) } diff --git a/internal/cli/tenant_test.go b/internal/cli/tenant_test.go new file mode 100644 index 0000000..5b66ba8 --- /dev/null +++ b/internal/cli/tenant_test.go @@ -0,0 +1,60 @@ +package cli + +import ( + "os" + "path/filepath" + "regexp" + "strings" + "testing" +) + +// tenant is the example tenant repository, which carries the instructions for coding agents. +const tenant = "../../examples/tenant" + +// shelfCommand finds a shelf command line in Markdown: in a code span or a code block. +var shelfCommand = regexp.MustCompile("(?m)(?:`|^\\s*)(shelf [a-z][^`\\n]*)") + +// TestTenantAgentInstructions keeps the skill honest: every shelf command it tells an agent to +// run exists, with the flags it names. +func TestTenantAgentInstructions(t *testing.T) { + t.Parallel() + root := New(Options{}) + seen := 0 + for _, file := range []string{".claude/skills/shelf/SKILL.md", "CLAUDE.md"} { + data, err := os.ReadFile(filepath.Join(tenant, file)) + if err != nil { + t.Fatal(err) + } + for _, m := range shelfCommand.FindAllStringSubmatch(string(data), -1) { + fields := strings.Fields(m[1])[1:] + cmd, rest, err := root.Find(fields) + if err != nil || cmd == root { + t.Errorf("%s: %q is not a shelf command", file, m[1]) + continue + } + for _, arg := range rest { + name, _, _ := strings.Cut(strings.TrimLeft(arg, "-"), "=") + if strings.HasPrefix(arg, "--") && cmd.Flags().Lookup(name) == nil { + t.Errorf("%s: %q: shelf %s has no flag --%s", file, m[1], cmd.Name(), name) + } + } + seen++ + } + } + if seen == 0 { + t.Fatal("the instructions name no shelf command") + } + + skill, _ := os.ReadFile(filepath.Join(tenant, ".claude/skills/shelf/SKILL.md")) + for _, want := range []string{`--wait --commit "$(git rev-parse HEAD)" --json`, "| 0 |", "| 1 |", "| 2 |", "Make the app public"} { + if !strings.Contains(string(skill), want) { + t.Errorf("the skill lacks %q", want) + } + } + // A hook would block the agent's turn for as long as a build takes. + for _, file := range []string{".claude/settings.json", ".claude/settings.local.json"} { + if _, err := os.Stat(filepath.Join(tenant, file)); err == nil { + t.Errorf("the tenant template ships %s", file) + } + } +} diff --git a/internal/cli/testdata/json/applist-empty.json b/internal/cli/testdata/json/applist-empty.json new file mode 100644 index 0000000..aeaece8 --- /dev/null +++ b/internal/cli/testdata/json/applist-empty.json @@ -0,0 +1,5 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "AppList", + "items": [] +} diff --git a/internal/cli/testdata/json/applist.json b/internal/cli/testdata/json/applist.json new file mode 100644 index 0000000..1d45b24 --- /dev/null +++ b/internal/cli/testdata/json/applist.json @@ -0,0 +1,24 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "AppList", + "items": [ + { + "name": "greeter", + "phase": "Failed", + "stage": "the pods", + "reason": "Back-off pulling image", + "artifact": "oci://ghcr.io/o/hello:main", + "revision": "main@sha256:abc", + "host": "greeter.shelf.internal" + }, + { + "name": "hello", + "phase": "Ready", + "artifact": "oci://ghcr.io/o/hello:main", + "revision": "main@sha256:abc", + "commit": "42a964eb0b4bd7248305d5f22233d6dbabcd56f3", + "host": "hello.shelf.internal", + "deployed": "2026-10-27T13:03:00Z" + } + ] +} diff --git a/internal/cli/testdata/json/appstatus-failing.json b/internal/cli/testdata/json/appstatus-failing.json new file mode 100644 index 0000000..d39a2e2 --- /dev/null +++ b/internal/cli/testdata/json/appstatus-failing.json @@ -0,0 +1,67 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "AppStatus", + "name": "hello", + "phase": "Failed", + "host": "hello.shelf.internal", + "artifact": "oci://ghcr.io/o/hello:main", + "registry": "ghcr", + "exposure": { + "kind": "none" + }, + "revision": "main@sha256:abc", + "commit": "42a964eb0b4bd7248305d5f22233d6dbabcd56f3", + "deployed": "2026-10-27T13:03:00Z", + "components": [ + { + "name": "db", + "phase": "Ready", + "address": "db:5432", + "internal": true + }, + { + "name": "web", + "phase": "Ready", + "path": "/", + "address": "hello.shelf.internal/" + } + ], + "stages": [ + { + "name": "the deploy artifact", + "object": "OCIRepository hello/deploy", + "phase": "Ready", + "reason": "Succeeded", + "message": "stored artifact for digest 'main@sha256:abc'" + }, + { + "name": "the app values", + "object": "Kustomization hello/deploy", + "phase": "Ready", + "reason": "ReconciliationSucceeded", + "message": "Applied revision: main@sha256:abc" + }, + { + "name": "the chart", + "object": "OCIRepository hello/shelf-app", + "phase": "Ready", + "reason": "Succeeded", + "message": "stored artifact for digest '0.6.0@sha256:def'" + }, + { + "name": "the app", + "object": "HelmRelease hello/hello", + "phase": "Ready", + "reason": "UpgradeSucceeded", + "message": "Helm upgrade succeeded" + }, + { + "name": "the pods", + "object": "Pod hello/web-6d4f", + "phase": "Failed", + "reason": "ImagePullBackOff", + "message": "Back-off pulling image", + "hint": "the image cannot be pulled; check the reference and whether the cluster may read that registry" + } + ] +} diff --git a/internal/cli/testdata/json/appstatus.json b/internal/cli/testdata/json/appstatus.json new file mode 100644 index 0000000..526a8a6 --- /dev/null +++ b/internal/cli/testdata/json/appstatus.json @@ -0,0 +1,65 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "AppStatus", + "name": "hello", + "phase": "Ready", + "host": "hello.shelf.internal", + "artifact": "oci://ghcr.io/o/hello:main", + "registry": "ghcr", + "exposure": { + "kind": "none" + }, + "revision": "main@sha256:abc", + "commit": "42a964eb0b4bd7248305d5f22233d6dbabcd56f3", + "deployed": "2026-10-27T13:03:00Z", + "components": [ + { + "name": "db", + "phase": "Ready", + "address": "db:5432", + "internal": true + }, + { + "name": "web", + "phase": "Ready", + "path": "/", + "address": "hello.shelf.internal/" + } + ], + "stages": [ + { + "name": "the deploy artifact", + "object": "OCIRepository hello/deploy", + "phase": "Ready", + "reason": "Succeeded", + "message": "stored artifact for digest 'main@sha256:abc'" + }, + { + "name": "the app values", + "object": "Kustomization hello/deploy", + "phase": "Ready", + "reason": "ReconciliationSucceeded", + "message": "Applied revision: main@sha256:abc" + }, + { + "name": "the chart", + "object": "OCIRepository hello/shelf-app", + "phase": "Ready", + "reason": "Succeeded", + "message": "stored artifact for digest '0.6.0@sha256:def'" + }, + { + "name": "the app", + "object": "HelmRelease hello/hello", + "phase": "Ready", + "reason": "UpgradeSucceeded", + "message": "Helm upgrade succeeded" + }, + { + "name": "the pods", + "object": "Pods", + "phase": "Ready", + "message": "2 running" + } + ] +} diff --git a/internal/cli/testdata/json/connectionlist-unused.json b/internal/cli/testdata/json/connectionlist-unused.json new file mode 100644 index 0000000..f636d16 --- /dev/null +++ b/internal/cli/testdata/json/connectionlist-unused.json @@ -0,0 +1,12 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "ConnectionList", + "items": [ + { + "kind": "cloudflare", + "name": "home", + "account": "acc-1", + "apps": [] + } + ] +} diff --git a/internal/cli/testdata/json/connectionlist.json b/internal/cli/testdata/json/connectionlist.json new file mode 100644 index 0000000..bfb7f83 --- /dev/null +++ b/internal/cli/testdata/json/connectionlist.json @@ -0,0 +1,23 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "ConnectionList", + "items": [ + { + "kind": "registry", + "name": "ghcr", + "user": "o", + "registry": "ghcr.io", + "apps": [ + "greeter" + ] + }, + { + "kind": "cloudflare", + "name": "home", + "account": "acc-1", + "apps": [ + "greeter" + ] + } + ] +} diff --git a/internal/cli/testdata/json/taglist.json b/internal/cli/testdata/json/taglist.json new file mode 100644 index 0000000..73038e1 --- /dev/null +++ b/internal/cli/testdata/json/taglist.json @@ -0,0 +1,20 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "TagList", + "app": "greeter", + "items": [ + { + "name": "main", + "created": "2026-09-03T08:00:00Z", + "current": true + }, + { + "name": "sha-333", + "created": "2026-09-03T08:00:00Z" + }, + { + "name": "sha-111", + "created": "2026-09-01T08:00:00Z" + } + ] +} diff --git a/internal/cli/testdata/json/version.json b/internal/cli/testdata/json/version.json new file mode 100644 index 0000000..ef52b64 --- /dev/null +++ b/internal/cli/testdata/json/version.json @@ -0,0 +1,6 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "Version", + "version": "v0.0.0-test", + "appAPIVersion": "shelf.dev/v1alpha1" +} diff --git a/internal/cli/testdata/json/waitresult-failed.json b/internal/cli/testdata/json/waitresult-failed.json new file mode 100644 index 0000000..ef36a3c --- /dev/null +++ b/internal/cli/testdata/json/waitresult-failed.json @@ -0,0 +1,16 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "WaitResult", + "outcome": "Failed", + "app": "hello", + "commit": "42a964eb0b4bd7248305d5f22233d6dbabcd56f3", + "revision": "main@sha256:abc", + "stage": { + "name": "the pods", + "object": "Pod hello/web-6d4f", + "phase": "Failed", + "reason": "ImagePullBackOff", + "message": "Back-off pulling image", + "hint": "the image cannot be pulled; check the reference and whether the cluster may read that registry" + } +} diff --git a/internal/cli/testdata/json/waitresult-ready.json b/internal/cli/testdata/json/waitresult-ready.json new file mode 100644 index 0000000..f4fd40b --- /dev/null +++ b/internal/cli/testdata/json/waitresult-ready.json @@ -0,0 +1,8 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "WaitResult", + "outcome": "Ready", + "app": "hello", + "commit": "42a964eb0b4bd7248305d5f22233d6dbabcd56f3", + "revision": "main@sha256:abc" +} diff --git a/internal/cli/testdata/json/waitresult-timedout.json b/internal/cli/testdata/json/waitresult-timedout.json new file mode 100644 index 0000000..10f1eb0 --- /dev/null +++ b/internal/cli/testdata/json/waitresult-timedout.json @@ -0,0 +1,14 @@ +{ + "apiVersion": "shelf.dev/v1alpha1", + "kind": "WaitResult", + "outcome": "TimedOut", + "app": "hello", + "commit": "9956993c371777310050f0c10d4f9b90aebcb969", + "revision": "main@sha256:old", + "stage": { + "name": "the deploy artifact", + "object": "OCIRepository hello/deploy", + "phase": "Working", + "message": "waiting for commit 42a964e; running 9956993" + } +} diff --git a/internal/cli/wait_test.go b/internal/cli/wait_test.go new file mode 100644 index 0000000..4fa1ad7 --- /dev/null +++ b/internal/cli/wait_test.go @@ -0,0 +1,109 @@ +package cli + +import ( + "strings" + "testing" + + "github.com/tweinmann/shelf/internal/cluster" +) + +// rolledOut is hello after the artifact of testCommit rolled out. +func rolledOut(pods cluster.Stage) cluster.Rollout { + return cluster.Rollout{ + Diagnosis: cluster.Diagnosis{Name: "hello", Stages: fiveStages(pods)}, + Revision: "main@sha256:abc", Commit: testCommit, + AppliedRevision: "main@sha256:abc", AttemptedRevision: "main@sha256:abc", + ValuesDigest: "sha256:values", ReleasedDigest: "sha256:values", AttemptedDigest: "sha256:values", + } +} + +func waitHarness(t *testing.T, rollouts ...cluster.Rollout) *harness { + t.Helper() + h := newHarness(t).public() + h.cluster.states = []cluster.AppState{helloState(cluster.PhaseReady)} + h.cluster.rollouts = rollouts + return h +} + +func TestWaitVerdicts(t *testing.T) { + t.Parallel() + previous := rolledOut(podsReady) + previous.Commit, previous.Revision = "9956993c371777310050f0c10d4f9b90aebcb969", "main@sha256:old" + tests := []struct { + name string + rollout cluster.Rollout + args []string + code int + text []string + }{ + {name: "waitresult-ready", rollout: rolledOut(podsReady), args: []string{"--commit", "42a964e"}, code: 0, + text: []string{"hello: Ready, running commit 42a964eb0b4b\n"}}, + {name: "waitresult-failed", rollout: rolledOut(podsFailed), code: 1, text: []string{ + "hello: Failed, running commit 42a964eb0b4b\n", + " the pods: ImagePullBackOff: Back-off pulling image\n", + " the image cannot be pulled; check the reference and whether the cluster may read that registry\n", + }}, + {name: "waitresult-timedout", rollout: previous, args: []string{"--commit", "42a964e", "--timeout", "20ms"}, code: 2, + text: []string{"hello: TimedOut, running commit 9956993c3717\n", + " the deploy artifact: waiting for commit 42a964e; running 9956993\n"}}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + args := append([]string{"app", "status", "hello", "--wait"}, tt.args...) + goldenJSON(t, tt.name, runJSON(t, waitHarness(t, tt.rollout), tt.code, args...)) + + stdout, stderr, code := waitHarness(t, tt.rollout).run(t, args...) + if code != tt.code { + t.Fatalf("exit code %d, want %d: %s", code, tt.code, stderr) + } + if want := strings.Join(tt.text, ""); stdout != want { + t.Errorf("stdout:\n%s\nwant:\n%s", stdout, want) + } + if stderr != "" { + t.Errorf("a verdict is not an error, but stderr says %q", stderr) + } + }) + } +} + +func TestWaitFollowsTheRollout(t *testing.T) { + t.Parallel() + arriving := rolledOut(podsReady) + arriving.Diagnosis.Stages[3].Phase = cluster.PhaseWorking + h := waitHarness(t, arriving, rolledOut(podsReady)) + h.mustRun(t, "app", "status", "hello", "--wait") + if h.cluster.rolloutCalls != 2 { + t.Errorf("read the rollout %d times, want 2", h.cluster.rolloutCalls) + } + if len(h.cluster.added)+len(h.cluster.removed)+len(h.cluster.redeployed) != 0 { + t.Errorf("waiting changed the cluster") + } +} + +func TestWaitRefusals(t *testing.T) { + t.Parallel() + tests := map[string]struct { + args []string + want string + }{ + "short commit": {[]string{"hello", "--wait", "--commit", "42a96"}, "at least 7 hex digits"}, + "commit not hex": {[]string{"hello", "--wait", "--commit", "main-branch"}, "is not a git commit"}, + "commit, no --wait": {[]string{"hello", "--commit", "42a964e"}, "--commit needs --wait"}, + "no app": {[]string{"--wait"}, "--wait needs the name of an app"}, + "unknown app": {[]string{"nope", "--wait"}, "app nope does not exist"}, + } + for name, tt := range tests { + t.Run(name, func(t *testing.T) { + t.Parallel() + h := waitHarness(t, rolledOut(podsReady)) + _, stderr, code := h.run(t, append([]string{"app", "status"}, tt.args...)...) + if code != 1 || !strings.Contains(stderr, tt.want) { + t.Errorf("exit code %d, stderr %q, want %q", code, stderr, tt.want) + } + if h.cluster.rolloutCalls != 0 && name != "unknown app" { + t.Errorf("waited %d times before refusing", h.cluster.rolloutCalls) + } + }) + } +} diff --git a/internal/cluster/forward.go b/internal/cluster/forward.go new file mode 100644 index 0000000..f29ca44 --- /dev/null +++ b/internal/cluster/forward.go @@ -0,0 +1,179 @@ +package cluster + +import ( + "context" + "errors" + "fmt" + "io" + "net/http" + "net/url" + "path" + "strconv" + + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/httpstream" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/portforward" + "k8s.io/client-go/transport/spdy" +) + +var ( + serviceGVK = schema.GroupVersionKind{Version: "v1", Kind: "Service"} + endpointSliceGVK = schema.GroupVersionKind{Group: "discovery.k8s.io", Version: "v1", Kind: "EndpointSlice"} +) + +const ( + // TraefikNamespace and traefikService are where the platform runs Traefik. + TraefikNamespace = "traefik" + traefikService = "traefik" + // traefikPort is the Service port every tunnel's cloudflared talks to. + traefikPort = 80 +) + +// Forward is a port-forward through the API server to a Traefik pod, listening on a loopback +// port of this machine. It ends when the pod goes away or the connection breaks; Done says so. +type Forward struct { + // Addr is the local end, 127.0.0.1:. + Addr string + Done <-chan struct{} + stop chan struct{} +} + +// Close ends the forward. +func (f *Forward) Close() { + if f.stop == nil { + return + } + select { + case <-f.stop: + default: + close(f.stop) + } +} + +// ForwardTraefik opens a port-forward to a ready Traefik pod, the same entry point a tunnel +// reaches through the Service, on a free loopback port. It talks to the API server only. +func ForwardTraefik(ctx context.Context, cfg *rest.Config) (*Forward, error) { + c, err := newClient(cfg) + if err != nil { + return nil, err + } + svc, err := c.get(ctx, ref{gvk: serviceGVK, namespace: TraefikNamespace, name: traefikService}) + if err != nil { + return nil, err + } + if svc == nil { + return nil, errors.New("the cluster has no Traefik service; run `shelf init cluster`") + } + slices, err := c.list(ctx, endpointSliceGVK, TraefikNamespace, "kubernetes.io/service-name="+traefikService) + if err != nil { + return nil, err + } + pod, port, err := traefikTarget(svc, slices) + if err != nil { + return nil, err + } + dialer, err := portForwardDialer(cfg, TraefikNamespace, pod) + if err != nil { + return nil, err + } + stop, ready := make(chan struct{}), make(chan struct{}) + fw, err := portforward.NewOnAddresses(dialer, []string{"127.0.0.1"}, []string{"0:" + strconv.Itoa(port)}, + stop, ready, io.Discard, io.Discard) + if err != nil { + return nil, err + } + done := make(chan struct{}) + failed := make(chan error, 1) + go func() { + defer close(done) + failed <- fw.ForwardPorts() + }() + f := &Forward{Done: done, stop: stop} + select { + case <-ready: + case err := <-failed: + return nil, fmt.Errorf("forwarding to pod %s/%s: %w", TraefikNamespace, pod, err) + case <-ctx.Done(): + f.Close() + return nil, ctx.Err() + } + ports, err := fw.GetPorts() + if err != nil || len(ports) == 0 { + f.Close() + return nil, fmt.Errorf("forwarding to pod %s/%s: no local port", TraefikNamespace, pod) + } + f.Addr = "127.0.0.1:" + strconv.Itoa(int(ports[0].Local)) + return f, nil +} + +// traefikTarget picks a ready pod behind the Traefik Service and the container port its port +// 80 leads to, as the Service itself would. +func traefikTarget(svc *unstructured.Unstructured, slices []*unstructured.Unstructured) (string, int, error) { + ports, _, _ := unstructured.NestedSlice(svc.Object, "spec", "ports") + name, found := "", false + for _, item := range ports { + p, _ := item.(map[string]any) + if number, _ := p["port"].(int64); number == traefikPort { + name, _ = p["name"].(string) + found = true + } + } + if !found { + return "", 0, fmt.Errorf("the Traefik service has no port %d", traefikPort) + } + for _, slice := range slices { + port := 0 + items, _, _ := unstructured.NestedSlice(slice.Object, "ports") + for _, item := range items { + p, _ := item.(map[string]any) + if n, _ := p["name"].(string); n == name { + number, _ := p["port"].(int64) + port = int(number) + } + } + if port == 0 { + continue + } + endpoints, _, _ := unstructured.NestedSlice(slice.Object, "endpoints") + for _, item := range endpoints { + e, _ := item.(map[string]any) + if ready, found, _ := unstructured.NestedBool(e, "conditions", "ready"); found && !ready { + continue + } + if kind, _, _ := unstructured.NestedString(e, "targetRef", "kind"); kind != "" && kind != "Pod" { + continue + } + if pod, _, _ := unstructured.NestedString(e, "targetRef", "name"); pod != "" { + return pod, port, nil + } + } + } + return "", 0, errors.New("no Traefik pod is ready; `shelf app status` and the pods in namespace traefik say why") +} + +// portForwardDialer connects to a pod's portforward subresource, over WebSocket where the API +// server supports it and over SPDY otherwise, as kubectl does. +func portForwardDialer(cfg *rest.Config, namespace, pod string) (httpstream.Dialer, error) { + host, err := url.Parse(cfg.Host) + if err != nil { + return nil, err + } + if host.Scheme == "" { + host.Scheme = "https" + } + host.Path = path.Join(host.Path, "/api/v1/namespaces", namespace, "pods", pod, "portforward") + transport, upgrader, err := spdy.RoundTripperFor(cfg) + if err != nil { + return nil, err + } + viaSPDY := spdy.NewDialer(upgrader, &http.Client{Transport: transport}, http.MethodPost, host) + viaWebSocket, err := portforward.NewSPDYOverWebsocketDialer(host, cfg) + if err != nil { + return nil, err + } + return portforward.NewFallbackDialer(viaWebSocket, viaSPDY, func(err error) bool { + return httpstream.IsUpgradeFailure(err) || httpstream.IsHTTPSProxyError(err) + }), nil +} diff --git a/internal/cluster/forward_test.go b/internal/cluster/forward_test.go new file mode 100644 index 0000000..893d635 --- /dev/null +++ b/internal/cluster/forward_test.go @@ -0,0 +1,78 @@ +package cluster + +import ( + "strings" + "testing" + + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" +) + +func serviceWithPort(port int64, name string) *unstructured.Unstructured { + return &unstructured.Unstructured{Object: map[string]any{"spec": map[string]any{"ports": []any{ + map[string]any{"name": name, "port": port, "targetPort": name}, + }}}} +} + +func endpointSlice(portName string, port int64, endpoints ...map[string]any) *unstructured.Unstructured { + items := make([]any, 0, len(endpoints)) + for _, e := range endpoints { + items = append(items, e) + } + return &unstructured.Unstructured{Object: map[string]any{ + "ports": []any{map[string]any{"name": portName, "port": port}}, + "endpoints": items, + }} +} + +func endpoint(pod string, ready bool) map[string]any { + return map[string]any{ + "conditions": map[string]any{"ready": ready}, + "targetRef": map[string]any{"kind": "Pod", "name": pod}, + } +} + +func TestTraefikTarget(t *testing.T) { + t.Parallel() + tests := []struct { + name string + svc *unstructured.Unstructured + slices []*unstructured.Unstructured + pod string + port int + err string + }{ + {name: "named target port", svc: serviceWithPort(80, "web"), + slices: []*unstructured.Unstructured{endpointSlice("web", 8000, endpoint("traefik-1", true))}, + pod: "traefik-1", port: 8000}, + {name: "unnamed port", svc: serviceWithPort(80, ""), + slices: []*unstructured.Unstructured{endpointSlice("", 8000, endpoint("traefik-1", true))}, + pod: "traefik-1", port: 8000}, + {name: "skips a pod that is not ready", svc: serviceWithPort(80, "web"), + slices: []*unstructured.Unstructured{endpointSlice("web", 8000, + endpoint("traefik-old", false), endpoint("traefik-new", true))}, + pod: "traefik-new", port: 8000}, + {name: "no ready pod", svc: serviceWithPort(80, "web"), + slices: []*unstructured.Unstructured{endpointSlice("web", 8000, endpoint("traefik-1", false))}, + err: "no Traefik pod is ready"}, + {name: "no endpoints", svc: serviceWithPort(80, "web"), err: "no Traefik pod is ready"}, + {name: "service without port 80", svc: serviceWithPort(443, "websecure"), err: "has no port 80"}, + {name: "slice for another port", svc: serviceWithPort(80, "web"), + slices: []*unstructured.Unstructured{endpointSlice("websecure", 8443, endpoint("traefik-1", true))}, + err: "no Traefik pod is ready"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + pod, port, err := traefikTarget(tt.svc, tt.slices) + if tt.err != "" { + if err == nil || !strings.Contains(err.Error(), tt.err) { + t.Fatalf("error %v, want %q", err, tt.err) + } + return + } + if err != nil || pod != tt.pod || port != tt.port { + t.Errorf("got %s:%d %v, want %s:%d", pod, port, err, tt.pod, tt.port) + } + }) + } +} diff --git a/internal/cluster/rollout.go b/internal/cluster/rollout.go new file mode 100644 index 0000000..547aea5 --- /dev/null +++ b/internal/cluster/rollout.go @@ -0,0 +1,126 @@ +package cluster + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "fmt" + + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/client-go/rest" + "sigs.k8s.io/yaml" + + "github.com/tweinmann/shelf/internal/render" +) + +// Rollout is an app's chain together with which artifact and which values each step is about. +// A Ready condition only says that a controller finished its last attempt; whether that attempt +// was about the newest artifact takes these fields. +type Rollout struct { + Diagnosis Diagnosis + // Revision and Commit are those of the deploy artifact the OCIRepository holds. + Revision string + Commit string + // AppliedRevision is the artifact revision the Kustomization last applied, AttemptedRevision + // the one it last tried; a failure is about the artifact only when it tried that one. + AppliedRevision string + AttemptedRevision string + // ValuesDigest is the digest of the values the release has to run now, computed the way + // helm-controller computes configDigest; empty while the values cannot be read. + ValuesDigest string + // ReleasedDigest is the configDigest of the release's latest Helm revision, AttemptedDigest + // that of the values helm-controller last started an attempt with. + ReleasedDigest string + AttemptedDigest string + // Reconciling means helm-controller is in the middle of an install, upgrade, rollback or + // retry, so its Ready condition is about to change. + Reconciling bool +} + +// AppRollout reads one app's chain and its progress, without changing anything. +func AppRollout(ctx context.Context, cfg *rest.Config, app string) (Rollout, error) { + c, err := newClient(cfg) + if err != nil { + return Rollout{}, err + } + apps, err := c.appObjects(ctx) + if err != nil { + return Rollout{}, err + } + objs, ok := apps[app] + if !ok { + return Rollout{}, fmt.Errorf("app %s does not exist", app) + } + values, err := c.get(ctx, ref{gvk: configMapGVK, namespace: app, name: render.ConfigMapName}) + if err != nil { + return Rollout{}, err + } + return rollout(app, objs, values), nil +} + +func rollout(app string, o *appObjects, values *unstructured.Unstructured) Rollout { + r := Rollout{Diagnosis: diagnose(app, o)} + if o.source != nil { + r.Revision, _, _ = unstructured.NestedString(o.source.Object, "status", "artifact", "revision") + r.Commit = artifactCommit(o.source) + } + if o.kustomization != nil { + r.AppliedRevision, _, _ = unstructured.NestedString(o.kustomization.Object, "status", "lastAppliedRevision") + r.AttemptedRevision, _, _ = unstructured.NestedString(o.kustomization.Object, "status", "lastAttemptedRevision") + } + if o.release == nil { + return r + } + r.AttemptedDigest, _, _ = unstructured.NestedString(o.release.Object, "status", "lastAttemptedConfigDigest") + history, _, _ := unstructured.NestedSlice(o.release.Object, "status", "history") + if len(history) > 0 { + if entry, ok := history[0].(map[string]any); ok { + r.ReleasedDigest, _ = entry["configDigest"].(string) + } + } + if reconciling := condition(o.release, "Reconciling"); reconciling != nil { + r.Reconciling = reconciling["status"] == "True" + } + if values != nil { + data, _, _ := unstructured.NestedString(values.Object, "data", render.ConfigMapKey) + inline, _, _ := unstructured.NestedMap(o.release.Object, "spec", "values") + r.ValuesDigest, _ = ConfigDigest(data, inline) + } + return r +} + +// ConfigDigest is helm-controller's configDigest of a release: the values from the ConfigMap, +// with the HelmRelease's inline values merged on top, as YAML, hashed with SHA-256. It is what +// tells a release that already runs the newest values from one that still runs older ones, +// which neither its conditions nor its timestamps can after a failed upgrade and its rollback. +func ConfigDigest(valuesYAML string, inline map[string]any) (string, error) { + values := map[string]any{} + if err := yaml.Unmarshal([]byte(valuesYAML), &values); err != nil { + return "", err + } + data, err := yaml.Marshal(mergeValues(values, inline)) + if err != nil { + return "", err + } + sum := sha256.Sum256(data) + return "sha256:" + hex.EncodeToString(sum[:]), nil +} + +// mergeValues merges b into a as Helm merges values: maps merge key by key, anything else in b +// replaces what a has. +func mergeValues(a, b map[string]any) map[string]any { + out := make(map[string]any, len(a)+len(b)) + for k, v := range a { + out[k] = v + } + for k, v := range b { + if bm, ok := v.(map[string]any); ok { + if am, ok := out[k].(map[string]any); ok { + out[k] = mergeValues(am, bm) + continue + } + } + out[k] = v + } + return out +} diff --git a/internal/cluster/rollout_test.go b/internal/cluster/rollout_test.go new file mode 100644 index 0000000..2e99aa6 --- /dev/null +++ b/internal/cluster/rollout_test.go @@ -0,0 +1,97 @@ +package cluster + +import ( + "encoding/json" + "os" + "path/filepath" + "testing" + + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" +) + +// TestConfigDigest pins the digest to one helm-controller wrote: testdata/configdigest holds the +// values ConfigMap and the HelmRelease of an app on the dev cluster (2026-10-27), with the +// configDigest of its release. If a Flux update changes how helm-controller hashes values, this +// test fails before any wait waits forever. +func TestConfigDigest(t *testing.T) { + t.Parallel() + values, err := os.ReadFile(filepath.Join("testdata", "configdigest", "app.yaml")) + if err != nil { + t.Fatal(err) + } + data, err := os.ReadFile(filepath.Join("testdata", "configdigest", "release.json")) + if err != nil { + t.Fatal(err) + } + var release struct { + Values map[string]any `json:"values"` + ConfigDigest string `json:"configDigest"` + } + if err := json.Unmarshal(data, &release); err != nil { + t.Fatal(err) + } + got, err := ConfigDigest(string(values), release.Values) + if err != nil { + t.Fatal(err) + } + if got != release.ConfigDigest { + t.Errorf("digest %s, helm-controller wrote %s", got, release.ConfigDigest) + } + other, _ := ConfigDigest(string(values)+"\nextra: 1\n", release.Values) + if other == got { + t.Error("other values have the same digest") + } +} + +func TestMergeValues(t *testing.T) { + t.Parallel() + got := mergeValues( + map[string]any{"name": "agent", "platform": map[string]any{"domain": "x", "keep": true}, "list": []any{1}}, + map[string]any{"name": "app", "platform": map[string]any{"domain": "shelf.internal"}, "list": []any{2}}, + ) + want := `{"list":[2],"name":"app","platform":{"domain":"shelf.internal","keep":true}}` + if b, _ := json.Marshal(got); string(b) != want { + t.Errorf("merged %s, want %s", b, want) + } +} + +func TestRollout(t *testing.T) { + t.Parallel() + objects := healthy() + _ = unstructured.SetNestedMap(objects.source.Object, map[string]any{ + "revision": "main@sha256:new", + "metadata": map[string]any{revisionAnnotation: "main@sha1:42a964eb0b4bd7248305d5f22233d6dbabcd56f3"}, + }, "status", "artifact") + _ = unstructured.SetNestedField(objects.kustomization.Object, "main@sha256:old", "status", "lastAppliedRevision") + _ = unstructured.SetNestedField(objects.kustomization.Object, "main@sha256:new", "status", "lastAttemptedRevision") + _ = unstructured.SetNestedMap(objects.release.Object, map[string]any{"name": "shop"}, "spec", "values") + _ = unstructured.SetNestedField(objects.release.Object, "sha256:attempted", "status", "lastAttemptedConfigDigest") + _ = unstructured.SetNestedSlice(objects.release.Object, []any{ + map[string]any{"version": int64(3), "configDigest": "sha256:released"}, + map[string]any{"version": int64(2), "configDigest": "sha256:older"}, + }, "status", "history") + conditions, _, _ := unstructured.NestedSlice(objects.release.Object, "status", "conditions") + conditions = append(conditions, map[string]any{"type": "Reconciling", "status": "True", "reason": "Progressing"}) + _ = unstructured.SetNestedSlice(objects.release.Object, conditions, "status", "conditions") + values := &unstructured.Unstructured{Object: map[string]any{"data": map[string]any{"app.yaml": "name: shop-pkg\n"}}} + + r := rollout("shop", objects, values) + if r.Revision != "main@sha256:new" || r.AppliedRevision != "main@sha256:old" || r.AttemptedRevision != "main@sha256:new" || + r.Commit != "42a964eb0b4bd7248305d5f22233d6dbabcd56f3" { + t.Errorf("revisions %+v", r) + } + want, _ := ConfigDigest("name: shop-pkg\n", map[string]any{"name": "shop"}) + if r.ValuesDigest != want || r.ReleasedDigest != "sha256:released" || r.AttemptedDigest != "sha256:attempted" { + t.Errorf("digests values %s (want %s), released %s, attempted %s", r.ValuesDigest, want, r.ReleasedDigest, r.AttemptedDigest) + } + if !r.Reconciling { + t.Error("a release with Reconciling=True is not reconciling") + } + if len(r.Diagnosis.Stages) != 5 { + t.Errorf("stages %d", len(r.Diagnosis.Stages)) + } + + if r := rollout("shop", healthy(), nil); r.ValuesDigest != "" || r.Commit != "" || r.Reconciling { + t.Errorf("an app without values or annotations %+v", r) + } +} diff --git a/internal/cluster/state.go b/internal/cluster/state.go index 3e082ac..42edeb2 100644 --- a/internal/cluster/state.go +++ b/internal/cluster/state.go @@ -91,6 +91,9 @@ type AppState struct { Cloudflare string `json:"cloudflare,omitempty"` // Revision is the deploy artifact that is applied, e.g. main@sha256:1f4e…. Revision string `json:"revision,omitempty"` + // Commit is the git commit the deploy artifact was built from, empty when the artifact does + // not name one. + Commit string `json:"commit,omitempty"` // Deployed is when the release last changed. Deployed time.Time `json:"deployed,omitzero"` Phase Phase `json:"phase"` @@ -250,6 +253,7 @@ func appState(name string, o *appObjects) AppState { state.Name = name if o.source != nil { state.Revision, _, _ = unstructured.NestedString(o.source.Object, "status", "artifact", "revision") + state.Commit = artifactCommit(o.source) } if o.release != nil { state.Deployed = lastDeployed(o.release) @@ -266,6 +270,28 @@ func appState(name string, o *appObjects) AppState { return state } +// revisionAnnotation is where `flux push artifact --revision` records the commit an artifact was +// built from. Flux copies the artifact's annotations into the status of the OCIRepository, so +// the cluster knows the commit without asking the forge. +const revisionAnnotation = "org.opencontainers.image.revision" + +// gitSHA is a full git commit hash. +var gitSHA = regexp.MustCompile(`^[0-9a-f]{40}$`) + +// artifactCommit returns the commit the source's artifact was built from: the revision +// annotation in its `@sha1:` form, or a bare commit hash. Anything else names no +// commit, as an artifact pushed with `--source local` and no revision does not. +func artifactCommit(source *unstructured.Unstructured) string { + value, _, _ := unstructured.NestedString(source.Object, "status", "artifact", "metadata", revisionAnnotation) + if _, sha, found := strings.Cut(value, "@sha1:"); found { + value = sha + } + if !gitSHA.MatchString(value) { + return "" + } + return value +} + // diagnose walks the chain in the order things have to happen and reports every step. Walking // in order is what makes the result useful: the first step that is not ready is the cause, and // everything after it is a consequence. diff --git a/internal/cluster/state_test.go b/internal/cluster/state_test.go index cfa4ddf..6cd619a 100644 --- a/internal/cluster/state_test.go +++ b/internal/cluster/state_test.go @@ -272,3 +272,33 @@ func TestShortenedMessages(t *testing.T) { t.Errorf("stage %q", state.Stage) } } + +func TestArtifactCommit(t *testing.T) { + t.Parallel() + const sha = "42a964eb0b4bd7248305d5f22233d6dbabcd56f3" + tests := []struct { + name string + metadata map[string]any + want string + }{ + {"pushed by the workflow", map[string]any{revisionAnnotation: "main@sha1:" + sha}, sha}, + {"bare commit", map[string]any{revisionAnnotation: sha}, sha}, + {"pushed by hand", map[string]any{revisionAnnotation: "local"}, ""}, + {"short commit", map[string]any{revisionAnnotation: "main@sha1:42a964e"}, ""}, + {"no revision", map[string]any{"dev.shelf.package": "shop"}, ""}, + {"no metadata", nil, ""}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + source := fluxObject("True", "Succeeded", "stored artifact") + if tt.metadata != nil { + _ = unstructured.SetNestedMap(source.Object, + map[string]any{"revision": "main@sha256:abc", "metadata": tt.metadata}, "status", "artifact") + } + if got := artifactCommit(source); got != tt.want { + t.Errorf("commit %q, want %q", got, tt.want) + } + }) + } +} diff --git a/internal/cluster/testdata/configdigest/app.yaml b/internal/cluster/testdata/configdigest/app.yaml new file mode 100644 index 0000000..8dbed37 --- /dev/null +++ b/internal/cluster/testdata/configdigest/app.yaml @@ -0,0 +1,22 @@ +apiVersion: shelf.dev/v1alpha1 +name: agent +components: + api: + image: traefik/whoami:v1.11.0@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab + ports: + main: 80 + route: + path: /api + port: main + stripPrefix: true + instances: 1 + web: + image: traefik/whoami:v1.11.0@sha256:200689790a0a0ea48ca45992e0450bc26ccab5307375b41c84dfc4f2475937ab + env: + WHOAMI_NAME: web-v1 + ports: + main: 80 + route: + path: / + port: main + instances: 1 diff --git a/internal/cluster/testdata/configdigest/release.json b/internal/cluster/testdata/configdigest/release.json new file mode 100644 index 0000000..b7435dd --- /dev/null +++ b/internal/cluster/testdata/configdigest/release.json @@ -0,0 +1,11 @@ +{ + "values": { + "name": "agent-spike", + "platform": { + "domain": "shelf.internal", + "hostSuffix": null, + "imagePullSecret": "shelf-registry" + } + }, + "configDigest": "sha256:2bea5f0e48691f804d2b51e3d1d591005fd30447c96df94ee481b7ff82a635cf" +} diff --git a/internal/ops/open.go b/internal/ops/open.go new file mode 100644 index 0000000..cf0c454 --- /dev/null +++ b/internal/ops/open.go @@ -0,0 +1,138 @@ +package ops + +import ( + "context" + "fmt" + "net" + "net/http" + "net/http/httputil" + "net/url" + "sync" + + "github.com/tweinmann/shelf/internal/cluster" +) + +// AppProxy carries requests from this machine to one app through a port-forward to Traefik. It +// sets the Host header to the app's own host, so Traefik routes them as it routes a tunnel's: +// the same routes, the same stripPrefix. Nothing about the app changes; it is not an exposure. +type AppProxy struct { + // Host is the app's own host, . or .shelf.internal. + Host string + + forward func(context.Context) (*cluster.Forward, error) + mu sync.Mutex + current *cluster.Forward + // replaced is called when a new forward takes over, to drop connections to the old one. + replaced func() +} + +// OpenApp checks that the app exists and opens the first forward, so that a cluster that +// cannot forward fails before anything listens. +func (o *Ops) OpenApp(ctx context.Context, name string) (*AppProxy, error) { + settings, err := o.Cluster.Settings(ctx, o.config()) + if err != nil { + return nil, err + } + states, err := o.Cluster.AppStates(ctx, o.config()) + if err != nil { + return nil, err + } + for _, state := range states { + if state.Name != name { + continue + } + // The app's own host, never a quick tunnel's random name: that is the name cloudflared + // sets too, and the one the app's Ingress answers to. + p := &AppProxy{ + Host: AppHost(name, settings.HostSuffix, state.Domain), + forward: func(ctx context.Context) (*cluster.Forward, error) { return o.Cluster.ForwardTraefik(ctx, o.config()) }, + } + if _, err := p.addr(ctx); err != nil { + return nil, err + } + return p, nil + } + return nil, &NotFoundError{Name: name} +} + +// NewAppProxy is an AppProxy over forwards made by forward, for tests. +func NewAppProxy(host string, forward func(context.Context) (*cluster.Forward, error)) *AppProxy { + return &AppProxy{Host: host, forward: forward} +} + +// Close ends the current forward. +func (p *AppProxy) Close() { + p.mu.Lock() + defer p.mu.Unlock() + if p.current != nil { + p.current.Close() + p.current = nil + } +} + +// addr is the local end of a working forward, opening a new one when the last one ended, as it +// does when the Traefik pod is replaced. +func (p *AppProxy) addr(ctx context.Context) (string, error) { + p.mu.Lock() + defer p.mu.Unlock() + if p.current != nil { + select { + case <-p.current.Done: + p.current = nil + default: + return p.current.Addr, nil + } + } + f, err := p.forward(ctx) + if err != nil { + return "", err + } + p.current = f + if p.replaced != nil { + p.replaced() + } + return f.Addr, nil +} + +// Handler serves the app to a browser that reaches it at local, e.g. 127.0.0.1:8081. +func (p *AppProxy) Handler(local string) http.Handler { + dialer := &net.Dialer{} + transport := http.DefaultTransport.(*http.Transport).Clone() + transport.Proxy = nil + transport.DialContext = func(ctx context.Context, network, _ string) (net.Conn, error) { + addr, err := p.addr(ctx) + if err != nil { + return nil, err + } + return dialer.DialContext(ctx, network, addr) + } + p.mu.Lock() + p.replaced = transport.CloseIdleConnections + p.mu.Unlock() + return &httputil.ReverseProxy{ + Transport: transport, + Rewrite: func(r *httputil.ProxyRequest) { + r.Out.URL.Scheme, r.Out.URL.Host = "http", p.Host + r.Out.Host = p.Host + // X-Forwarded-Host is where the browser is, so an app that builds links from it + // sends the browser back here. + r.SetXForwarded() + }, + ModifyResponse: func(resp *http.Response) error { + location := resp.Header.Get("Location") + if location == "" { + return nil + } + u, err := url.Parse(location) + if err != nil || u.Hostname() != p.Host { + return nil + } + u.Scheme, u.Host = "http", local + resp.Header.Set("Location", u.String()) + return nil + }, + ErrorHandler: func(w http.ResponseWriter, _ *http.Request, err error) { + http.Error(w, fmt.Sprintf("shelf cannot reach the cluster's Traefik: %v", err), http.StatusBadGateway) + }, + } +} diff --git a/internal/ops/open_test.go b/internal/ops/open_test.go new file mode 100644 index 0000000..2af2470 --- /dev/null +++ b/internal/ops/open_test.go @@ -0,0 +1,202 @@ +package ops_test + +import ( + "bufio" + "context" + "errors" + "fmt" + "io" + "net" + "net/http" + "net/http/httptest" + "strings" + "sync" + "testing" + + "github.com/tweinmann/shelf/internal/cluster" + "github.com/tweinmann/shelf/internal/ops" +) + +// fakeTraefik answers like Traefik in front of an app: it reports the Host and headers it got, +// redirects /login to the app's own host, and switches /ws to an echo protocol. +func fakeTraefik(t *testing.T) *httptest.Server { + t.Helper() + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + switch r.URL.Path { + case "/login": + http.Redirect(w, r, "http://hello-dev.example.com/login/form?next=%2F", http.StatusFound) + case "/elsewhere": + http.Redirect(w, r, "https://accounts.example.org/", http.StatusFound) + case "/ws": + conn, buf, err := http.NewResponseController(w).Hijack() + if err != nil { + t.Error(err) + return + } + defer conn.Close() + fmt.Fprint(buf, "HTTP/1.1 101 Switching Protocols\r\nConnection: Upgrade\r\nUpgrade: echo\r\n\r\n") + _ = buf.Flush() + line, _ := buf.ReadString('\n') + fmt.Fprint(buf, "echo: "+line) + _ = buf.Flush() + default: + fmt.Fprintf(w, "host=%s path=%s fwd-host=%s fwd-proto=%s", r.Host, r.URL.Path, + r.Header.Get("X-Forwarded-Host"), r.Header.Get("X-Forwarded-Proto")) + } + })) + t.Cleanup(srv.Close) + return srv +} + +// forwards hands out forwards to Traefik and counts them. Each one is a relay on a port of its +// own that end closes with every connection through it, as a broken port-forward does. +type forwards struct { + traefik string + mu sync.Mutex + made int + relay net.Listener + conns []net.Conn + done chan struct{} + broke bool +} + +func (f *forwards) open(context.Context) (*cluster.Forward, error) { + f.mu.Lock() + defer f.mu.Unlock() + if f.broke { + return nil, errors.New("no Traefik pod is ready") + } + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + return nil, err + } + f.made++ + f.relay, f.done = ln, make(chan struct{}) + go func() { + for { + in, err := ln.Accept() + if err != nil { + return + } + out, err := net.Dial("tcp", f.traefik) + if err != nil { + in.Close() + return + } + f.mu.Lock() + f.conns = append(f.conns, in, out) + f.mu.Unlock() + go func() { _, _ = io.Copy(out, in); out.Close() }() + go func() { _, _ = io.Copy(in, out); in.Close() }() + } + }() + return &cluster.Forward{Addr: ln.Addr().String(), Done: f.done}, nil +} + +func (f *forwards) end() { + f.mu.Lock() + defer f.mu.Unlock() + f.relay.Close() + for _, c := range f.conns { + c.Close() + } + f.conns = nil + close(f.done) +} + +func proxy(t *testing.T, f *forwards) (*httptest.Server, string) { + t.Helper() + p := ops.NewAppProxy("hello-dev.example.com", f.open) + // The local server stands for the listener shelf app open binds; its address is the one + // the browser sees. + local := httptest.NewUnstartedServer(nil) + local.Config.Handler = p.Handler(local.Listener.Addr().String()) + local.Start() + t.Cleanup(local.Close) + t.Cleanup(p.Close) + return local, local.Listener.Addr().String() +} + +func get(t *testing.T, url string) (*http.Response, string) { + t.Helper() + client := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }} + resp, err := client.Get(url) + if err != nil { + t.Fatal(err) + } + defer resp.Body.Close() + body, _ := io.ReadAll(resp.Body) + return resp, string(body) +} + +func TestAppProxySetsTheAppsHost(t *testing.T) { + t.Parallel() + f := &forwards{traefik: fakeTraefik(t).Listener.Addr().String()} + local, addr := proxy(t, f) + _, body := get(t, local.URL+"/api/x") + want := "host=hello-dev.example.com path=/api/x fwd-host=" + addr + " fwd-proto=http" + if body != want { + t.Errorf("Traefik got %q, want %q", body, want) + } +} + +func TestAppProxyRewritesRedirects(t *testing.T) { + t.Parallel() + f := &forwards{traefik: fakeTraefik(t).Listener.Addr().String()} + local, addr := proxy(t, f) + resp, _ := get(t, local.URL+"/login") + if got, want := resp.Header.Get("Location"), "http://"+addr+"/login/form?next=%2F"; got != want { + t.Errorf("Location %q, want %q", got, want) + } + resp, _ = get(t, local.URL+"/elsewhere") + if got := resp.Header.Get("Location"); got != "https://accounts.example.org/" { + t.Errorf("a redirect to another host was changed to %q", got) + } +} + +func TestAppProxyPassesUpgrades(t *testing.T) { + t.Parallel() + f := &forwards{traefik: fakeTraefik(t).Listener.Addr().String()} + _, addr := proxy(t, f) + conn, err := net.Dial("tcp", addr) + if err != nil { + t.Fatal(err) + } + defer conn.Close() + fmt.Fprint(conn, "GET /ws HTTP/1.1\r\nHost: "+addr+"\r\nConnection: Upgrade\r\nUpgrade: echo\r\n\r\n") + r := bufio.NewReader(conn) + resp, err := http.ReadResponse(r, nil) + if err != nil || resp.StatusCode != http.StatusSwitchingProtocols { + t.Fatalf("upgrade: %v %v", resp, err) + } + fmt.Fprint(conn, "hello\n") + if line, _ := r.ReadString('\n'); line != "echo: hello\n" { + t.Errorf("after the upgrade: %q", line) + } +} + +func TestAppProxyReconnects(t *testing.T) { + t.Parallel() + f := &forwards{traefik: fakeTraefik(t).Listener.Addr().String()} + local, _ := proxy(t, f) + get(t, local.URL+"/") + f.end() + if _, body := get(t, local.URL+"/again"); !strings.Contains(body, "path=/again") { + t.Errorf("after the forward ended: %q", body) + } + f.mu.Lock() + made := f.made + f.mu.Unlock() + if made != 2 { + t.Errorf("opened %d forwards, want 2", made) + } + + f.end() + f.mu.Lock() + f.broke = true + f.mu.Unlock() + resp, body := get(t, local.URL+"/") + if resp.StatusCode != http.StatusBadGateway || !strings.Contains(body, "no Traefik pod is ready") { + t.Errorf("without a forward: %d %q", resp.StatusCode, body) + } +} diff --git a/internal/ops/ops.go b/internal/ops/ops.go index d025b7f..44b578d 100644 --- a/internal/ops/ops.go +++ b/internal/ops/ops.go @@ -88,6 +88,8 @@ type Cluster interface { AppStates(ctx context.Context, cfg *rest.Config) ([]cluster.AppState, error) AppComponents(ctx context.Context, cfg *rest.Config, app string) ([]cluster.Component, error) AppDiagnosis(ctx context.Context, cfg *rest.Config, app string) (cluster.Diagnosis, error) + AppRollout(ctx context.Context, cfg *rest.Config, app string) (cluster.Rollout, error) + ForwardTraefik(ctx context.Context, cfg *rest.Config) (*cluster.Forward, error) Settings(ctx context.Context, cfg *rest.Config) (cluster.Settings, error) // AppConfig returns how an app is registered, or nil when there is no such app. AppConfig(ctx context.Context, cfg *rest.Config, app string) (*cluster.AppConfig, error) @@ -144,6 +146,14 @@ func (liveCluster) AppDiagnosis(ctx context.Context, cfg *rest.Config, app strin return cluster.AppDiagnosis(ctx, cfg, app) } +func (liveCluster) AppRollout(ctx context.Context, cfg *rest.Config, app string) (cluster.Rollout, error) { + return cluster.AppRollout(ctx, cfg, app) +} + +func (liveCluster) ForwardTraefik(ctx context.Context, cfg *rest.Config) (*cluster.Forward, error) { + return cluster.ForwardTraefik(ctx, cfg) +} + func (liveCluster) Settings(ctx context.Context, cfg *rest.Config) (cluster.Settings, error) { return cluster.ClusterSettings(ctx, cfg) } diff --git a/internal/ops/wait.go b/internal/ops/wait.go new file mode 100644 index 0000000..11af6ec --- /dev/null +++ b/internal/ops/wait.go @@ -0,0 +1,195 @@ +package ops + +import ( + "context" + "errors" + "fmt" + "regexp" + "strings" + "time" + + "github.com/tweinmann/shelf/internal/cluster" +) + +// Outcome is how a wait for an app ended. +type Outcome string + +const ( + // OutcomeReady means the app, at the awaited commit if one was named, is running. + OutcomeReady Outcome = "Ready" + // OutcomeFailed means a stage failed; it will not fix itself. + OutcomeFailed Outcome = "Failed" + // OutcomeTimedOut means the app was still on its way when the time ran out. + OutcomeTimedOut Outcome = "TimedOut" +) + +// DefaultWaitTimeout covers a CI build of a few minutes plus the minute the platform takes to +// notice the new artifact. +const DefaultWaitTimeout = 10 * time.Minute + +// waitInterval is how often a wait reads the cluster. +const waitInterval = 2 * time.Second + +// WaitOptions say what to wait for. +type WaitOptions struct { + // Commit is the git commit, or a prefix of at least 7 hex digits, the app has to run. Empty + // means whatever artifact the app has. + Commit string + Timeout time.Duration + // Interval overrides how often the cluster is read, for tests. + Interval time.Duration +} + +// WaitResult is the verdict of a wait. +type WaitResult struct { + Outcome Outcome + App string + Commit string + Revision string + // Stage is the first stage that is not Ready, nil when the outcome is Ready. + Stage *cluster.Stage +} + +var commitPrefix = regexp.MustCompile(`^[0-9a-f]{7,40}$`) + +// CheckCommit checks the commit a wait is asked for. +func CheckCommit(commit string) error { + if !commitPrefix.MatchString(commit) { + return fmt.Errorf("%q is not a git commit: give at least 7 hex digits, as `git rev-parse HEAD` prints", commit) + } + return nil +} + +// Wait reads the app's chain until it is Ready or a stage failed, or the timeout ends. It only +// reads: asking Flux to hurry would make a status query a change to the cluster. +func (o *Ops) Wait(ctx context.Context, app string, opts WaitOptions) (WaitResult, error) { + if opts.Commit != "" { + if err := CheckCommit(opts.Commit); err != nil { + return WaitResult{}, err + } + } + interval := opts.Interval + if interval <= 0 { + interval = waitInterval + } + timeout := opts.Timeout + if timeout <= 0 { + timeout = DefaultWaitTimeout + } + ctx, cancel := context.WithTimeout(ctx, timeout) + defer cancel() + ticker := time.NewTicker(interval) + defer ticker.Stop() + var last WaitResult + for { + r, err := o.Cluster.AppRollout(ctx, o.config(), app) + switch { + case err == nil: + var done bool + if last, done = Verdict(app, r, opts.Commit); done { + return last, nil + } + case ctx.Err() == nil: + return WaitResult{}, err + } + select { + case <-ctx.Done(): + if errors.Is(ctx.Err(), context.DeadlineExceeded) && last.App != "" { + last.Outcome = OutcomeTimedOut + return last, nil + } + return WaitResult{}, ctx.Err() + case <-ticker.C: + } + } +} + +// Verdict decides what one reading of an app's rollout means. It reports false while the app +// is still on its way, together with the stage that is holding it up. +// +// The stages are walked in order, as the diagnosis does, with one addition: a stage reporting +// on an older artifact, Ready or Failed, is still on its way. The deploy artifact has to be the +// awaited commit, the values have to be those of that artifact, and the release has to hold +// those values. Before the awaited commit has arrived, only a failing deploy artifact ends the +// wait: the later stages are still about the previous one. +func Verdict(app string, r cluster.Rollout, commit string) (WaitResult, bool) { + result := WaitResult{App: app, Commit: r.Commit, Revision: r.Revision} + stages := r.Diagnosis.Stages + if len(stages) < 5 { + return result, false + } + failed := func(s cluster.Stage) (WaitResult, bool) { + result.Outcome, result.Stage = OutcomeFailed, &s + return result, true + } + pending := func(s cluster.Stage) (WaitResult, bool) { + result.Outcome, result.Stage = OutcomeTimedOut, &s + return result, false + } + working := func(s cluster.Stage, message string) (WaitResult, bool) { + s.Phase, s.Reason, s.Message, s.Hint = cluster.PhaseWorking, "", message, "" + return pending(s) + } + + source, values, chart, release, pods := stages[0], stages[1], stages[2], stages[3], stages[4] + switch { + case source.Phase == cluster.PhaseFailed: + return failed(source) + case source.Phase != cluster.PhaseReady: + return pending(source) + case commit != "" && r.Commit == "": + return working(source, "waiting for commit "+short(commit)+"; the deploy artifact names no commit") + case commit != "" && !strings.HasPrefix(r.Commit, commit): + return working(source, "waiting for commit "+short(commit)+"; running "+short(r.Commit)) + } + + switch { + case values.Phase == cluster.PhaseFailed && r.AttemptedRevision == r.Revision: + return failed(values) + case values.Phase == cluster.PhaseReady && r.AppliedRevision == r.Revision: + case values.Phase == cluster.PhaseMissing || values.Phase == cluster.PhaseWorking: + return pending(values) + default: + return working(values, "waiting for the values of "+r.Revision) + } + + switch chart.Phase { + case cluster.PhaseReady: + case cluster.PhaseFailed: + return failed(chart) + default: + return pending(chart) + } + + // Which values a release holds only its digest says: after a failed upgrade, the failure + // and the rollback both happen after newer values arrived, and still belong to the older. + current := r.ValuesDigest != "" && r.ReleasedDigest == r.ValuesDigest && !r.Reconciling + attempting := r.ValuesDigest != "" && r.AttemptedDigest == r.ValuesDigest + switch { + case current && release.Phase == cluster.PhaseReady: + case current && release.Phase == cluster.PhaseFailed: + return failed(release) + case attempting && release.Phase != cluster.PhaseReady: + // The release waits for its pods, so a pod that cannot start is known long before the + // release gives up on it. + if pods.Phase == cluster.PhaseFailed { + return failed(pods) + } + return pending(release) + default: + return working(release, "waiting for the release to pick up the new values") + } + + switch pods.Phase { + case cluster.PhaseReady: + case cluster.PhaseFailed: + return failed(pods) + default: + return pending(pods) + } + result.Outcome = OutcomeReady + return result, true +} + +// short is how a commit is named in a message. +func short(commit string) string { return commit[:min(len(commit), 7)] } diff --git a/internal/ops/wait_test.go b/internal/ops/wait_test.go new file mode 100644 index 0000000..ba98b8a --- /dev/null +++ b/internal/ops/wait_test.go @@ -0,0 +1,241 @@ +package ops_test + +import ( + "context" + "strings" + "sync" + "testing" + "time" + + "k8s.io/client-go/rest" + + "github.com/tweinmann/shelf/internal/cluster" + "github.com/tweinmann/shelf/internal/ops" +) + +const ( + oldCommit = "9956993c371777310050f0c10d4f9b90aebcb969" + newCommit = "42a964eb0b4bd7248305d5f22233d6dbabcd56f3" +) + +// The digests of the values: those of the awaited artifact, of the one before, and of a broken +// one in between. +const ( + newValues = "sha256:new-values" + oldValues = "sha256:old-values" + brokenValues = "sha256:broken-values" +) + +func stage(name string, phase cluster.Phase) cluster.Stage { + return cluster.Stage{Name: name, Object: "Object " + name, Phase: phase, Reason: "R", Message: "m", Hint: "h"} +} + +// settled is an app whose new artifact has rolled out completely. +func settled() cluster.Rollout { + return cluster.Rollout{ + Diagnosis: cluster.Diagnosis{Name: "hello", Stages: []cluster.Stage{ + stage("the deploy artifact", cluster.PhaseReady), stage("the app values", cluster.PhaseReady), + stage("the chart", cluster.PhaseReady), stage("the app", cluster.PhaseReady), + stage("the pods", cluster.PhaseReady), + }}, + Revision: "main@sha256:new", Commit: newCommit, + AppliedRevision: "main@sha256:new", AttemptedRevision: "main@sha256:new", + ValuesDigest: newValues, ReleasedDigest: newValues, AttemptedDigest: newValues, + } +} + +func with(r cluster.Rollout, change func(*cluster.Rollout)) cluster.Rollout { + r.Diagnosis.Stages = append([]cluster.Stage(nil), r.Diagnosis.Stages...) + change(&r) + return r +} + +func phase(i int, p cluster.Phase) func(*cluster.Rollout) { + return func(r *cluster.Rollout) { r.Diagnosis.Stages[i].Phase = p } +} + +func TestVerdict(t *testing.T) { + t.Parallel() + // previous is the app still running the artifact before the awaited one. + previous := func(r *cluster.Rollout) { + r.Commit, r.Revision, r.AppliedRevision, r.AttemptedRevision = oldCommit, "main@sha256:old", "main@sha256:old", "main@sha256:old" + } + tests := []struct { + name string + rollout cluster.Rollout + commit string + done bool + outcome ops.Outcome + stage string + phase cluster.Phase + contains string + }{ + {name: "already Ready", rollout: settled(), done: true, outcome: ops.OutcomeReady}, + {name: "Ready at the awaited commit", rollout: settled(), commit: "42a964e", done: true, outcome: ops.OutcomeReady}, + {name: "full commit", rollout: settled(), commit: newCommit, done: true, outcome: ops.OutcomeReady}, + {name: "pods starting", rollout: with(settled(), phase(4, cluster.PhaseWorking)), + stage: "the pods", phase: cluster.PhaseWorking}, + {name: "release upgrading", rollout: with(settled(), func(r *cluster.Rollout) { + r.ReleasedDigest, r.Reconciling = oldValues, true + phase(3, cluster.PhaseWorking)(r) + }), stage: "the app", phase: cluster.PhaseWorking, contains: "m"}, + {name: "previous commit still running", rollout: with(settled(), previous), commit: "42a964e", + stage: "the deploy artifact", phase: cluster.PhaseWorking, contains: "waiting for commit 42a964e; running 9956993"}, + {name: "previous artifact failed while waiting for the new one", + rollout: with(settled(), func(r *cluster.Rollout) { previous(r); phase(4, cluster.PhaseFailed)(r) }), + commit: "42a964e", stage: "the deploy artifact", phase: cluster.PhaseWorking}, + {name: "registry refuses the artifact", rollout: with(settled(), func(r *cluster.Rollout) { + previous(r) + phase(0, cluster.PhaseFailed)(r) + }), commit: "42a964e", done: true, outcome: ops.OutcomeFailed, stage: "the deploy artifact", phase: cluster.PhaseFailed}, + {name: "artifact without a commit", rollout: with(settled(), func(r *cluster.Rollout) { r.Commit = "" }), + commit: "42a964e", stage: "the deploy artifact", contains: "names no commit"}, + {name: "values of the previous artifact", rollout: with(settled(), func(r *cluster.Rollout) { + r.AppliedRevision, r.AttemptedRevision = "main@sha256:old", "main@sha256:old" + }), stage: "the app values", phase: cluster.PhaseWorking, contains: "waiting for the values of main@sha256:new"}, + {name: "values failed for the previous artifact", rollout: with(settled(), func(r *cluster.Rollout) { + r.AppliedRevision, r.AttemptedRevision = "main@sha256:older", "main@sha256:old" + phase(1, cluster.PhaseFailed)(r) + }), stage: "the app values", phase: cluster.PhaseWorking}, + {name: "values rejected for this artifact", rollout: with(settled(), func(r *cluster.Rollout) { + r.AppliedRevision = "main@sha256:old" + phase(1, cluster.PhaseFailed)(r) + }), done: true, outcome: ops.OutcomeFailed, stage: "the app values", phase: cluster.PhaseFailed}, + // The spike of 2026-10-27, step by step: a fix arrives while Helm still waits for the + // broken upgrade before it, then rolls it back and retries. + {name: "previous values still released", rollout: with(settled(), func(r *cluster.Rollout) { + r.ReleasedDigest, r.AttemptedDigest = oldValues, oldValues + }), stage: "the app", phase: cluster.PhaseWorking, contains: "pick up the new values"}, + {name: "broken upgrade before the fix still running", rollout: with(settled(), func(r *cluster.Rollout) { + r.ReleasedDigest, r.AttemptedDigest, r.Reconciling = oldValues, brokenValues, true + phase(3, cluster.PhaseWorking)(r) + phase(4, cluster.PhaseFailed)(r) + }), stage: "the app", phase: cluster.PhaseWorking, contains: "pick up the new values"}, + {name: "broken upgrade before the fix failed", rollout: with(settled(), func(r *cluster.Rollout) { + r.ReleasedDigest, r.AttemptedDigest, r.Reconciling = brokenValues, brokenValues, true + phase(3, cluster.PhaseFailed)(r) + }), stage: "the app", phase: cluster.PhaseWorking}, + {name: "rollback after the fix arrived", rollout: with(settled(), func(r *cluster.Rollout) { + r.ReleasedDigest, r.AttemptedDigest, r.Reconciling = oldValues, brokenValues, true + phase(3, cluster.PhaseFailed)(r) + }), stage: "the app", phase: cluster.PhaseWorking}, + {name: "fix released without a new revision", rollout: with(settled(), func(r *cluster.Rollout) { + r.ReleasedDigest, r.AttemptedDigest = newValues, newValues + }), done: true, outcome: ops.OutcomeReady}, + {name: "released, but helm-controller is reconciling again", rollout: with(settled(), func(r *cluster.Rollout) { + r.Reconciling = true + }), stage: "the app", phase: cluster.PhaseWorking}, + {name: "values cannot be read", rollout: with(settled(), func(r *cluster.Rollout) { r.ValuesDigest = "" }), + stage: "the app", phase: cluster.PhaseWorking}, + {name: "upgrade failed", rollout: with(settled(), phase(3, cluster.PhaseFailed)), + done: true, outcome: ops.OutcomeFailed, stage: "the app", phase: cluster.PhaseFailed}, + {name: "image cannot be pulled while the release waits", rollout: with(settled(), func(r *cluster.Rollout) { + r.ReleasedDigest, r.Reconciling = oldValues, true + phase(3, cluster.PhaseWorking)(r) + phase(4, cluster.PhaseFailed)(r) + }), done: true, outcome: ops.OutcomeFailed, stage: "the pods", phase: cluster.PhaseFailed}, + {name: "pods failing after the release", rollout: with(settled(), phase(4, cluster.PhaseFailed)), + done: true, outcome: ops.OutcomeFailed, stage: "the pods"}, + {name: "not created yet", rollout: with(settled(), phase(0, cluster.PhaseMissing)), + stage: "the deploy artifact", phase: cluster.PhaseMissing}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + got, done := ops.Verdict("hello", tt.rollout, tt.commit) + if done != tt.done { + t.Fatalf("done %v, want %v: %+v", done, tt.done, got) + } + if tt.done && got.Outcome != tt.outcome { + t.Errorf("outcome %s, want %s", got.Outcome, tt.outcome) + } + if tt.stage == "" { + if got.Stage != nil { + t.Errorf("stage %+v, want none", got.Stage) + } + return + } + if got.Stage == nil || got.Stage.Name != tt.stage { + t.Fatalf("stage %+v, want %s", got.Stage, tt.stage) + } + if tt.phase != "" && got.Stage.Phase != tt.phase { + t.Errorf("phase %s, want %s", got.Stage.Phase, tt.phase) + } + if !strings.Contains(got.Stage.Message, tt.contains) { + t.Errorf("message %q lacks %q", got.Stage.Message, tt.contains) + } + }) + } +} + +// rolloutCluster answers AppRollout from a script. Every other method of ops.Cluster is the nil +// interface's, so a wait that tried to change anything would panic: waiting is read-only. +type rolloutCluster struct { + ops.Cluster + mu sync.Mutex + rollouts []cluster.Rollout + calls int +} + +func (c *rolloutCluster) AppRollout(context.Context, *rest.Config, string) (cluster.Rollout, error) { + c.mu.Lock() + defer c.mu.Unlock() + r := c.rollouts[min(c.calls, len(c.rollouts)-1)] + c.calls++ + return r, nil +} + +func TestWait(t *testing.T) { + t.Parallel() + before := with(settled(), func(r *cluster.Rollout) { + r.Commit, r.Revision = oldCommit, "main@sha256:old" + }) + arrived := with(settled(), func(r *cluster.Rollout) { r.ReleasedDigest = oldValues }) + tests := []struct { + name string + rollouts []cluster.Rollout + commit string + timeout time.Duration + outcome ops.Outcome + calls int + }{ + {name: "until the commit rolled out", rollouts: []cluster.Rollout{before, before, arrived, settled()}, + commit: "42a964e", timeout: time.Minute, outcome: ops.OutcomeReady, calls: 4}, + {name: "timeout while the previous commit runs", rollouts: []cluster.Rollout{before}, + commit: "42a964e", timeout: 50 * time.Millisecond, outcome: ops.OutcomeTimedOut}, + {name: "failure ends the wait", rollouts: []cluster.Rollout{arrived, with(settled(), phase(4, cluster.PhaseFailed))}, + timeout: time.Minute, outcome: ops.OutcomeFailed, calls: 2}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + fake := &rolloutCluster{rollouts: tt.rollouts} + o := &ops.Ops{Target: ops.Target{Config: &rest.Config{}}, Cluster: fake} + got, err := o.Wait(context.Background(), "hello", + ops.WaitOptions{Commit: tt.commit, Timeout: tt.timeout, Interval: time.Millisecond}) + if err != nil { + t.Fatal(err) + } + if got.Outcome != tt.outcome { + t.Errorf("outcome %s, want %s: %+v", got.Outcome, tt.outcome, got.Stage) + } + if tt.calls != 0 && fake.calls != tt.calls { + t.Errorf("read the cluster %d times, want %d", fake.calls, tt.calls) + } + if tt.outcome == ops.OutcomeTimedOut && (got.Stage == nil || got.Stage.Name != "the deploy artifact") { + t.Errorf("a timeout names the stage it waited at: %+v", got.Stage) + } + }) + } +} + +func TestWaitRefusesABadCommit(t *testing.T) { + t.Parallel() + o := &ops.Ops{Cluster: &rolloutCluster{}} + for _, commit := range []string{"42a96", "not-hex", "42A964EB"} { + if _, err := o.Wait(context.Background(), "hello", ops.WaitOptions{Commit: commit}); err == nil || + !strings.Contains(err.Error(), "at least 7 hex digits") { + t.Errorf("%q: %v", commit, err) + } + } +} diff --git a/justfile b/justfile index 2e8ca32..eed4c08 100644 --- a/justfile +++ b/justfile @@ -100,6 +100,10 @@ smoke-init: smoke-apps: hack/smoke/apps.sh +# Phase 9: --json, --wait --commit (Ready, Failed, TimedOut, a fix after a failure) and shelf app open +smoke-agent: + hack/smoke/agent.sh + # Phase 8b acceptance: expose an app through its own Cloudflare tunnel (asks for the API token) smoke-expose app domain: hack/smoke/expose.sh {{app}} {{domain}} diff --git a/openspec/changes/phase-9-mac-mini/.openspec.yaml b/openspec/changes/phase-10-branch-previews/.openspec.yaml similarity index 64% rename from openspec/changes/phase-9-mac-mini/.openspec.yaml rename to openspec/changes/phase-10-branch-previews/.openspec.yaml index 9459bc2..ad9e5be 100644 --- a/openspec/changes/phase-9-mac-mini/.openspec.yaml +++ b/openspec/changes/phase-10-branch-previews/.openspec.yaml @@ -1,3 +1,3 @@ schema: spec-driven -created: 2026-10-20 +created: 2026-10-27 skip_specs: true diff --git a/openspec/changes/phase-10-branch-previews/proposal.md b/openspec/changes/phase-10-branch-previews/proposal.md new file mode 100644 index 0000000..5c62b6a --- /dev/null +++ b/openspec/changes/phase-10-branch-previews/proposal.md @@ -0,0 +1,66 @@ +# Proposal + +> Draft. This change holds the roadmap text only and is marked `skip_specs: true`. +> `/opsx:propose` turns it into a full change: it writes the delta specs, `design.md` and +> `tasks.md`, revises this proposal, and removes `skip_specs` from `.openspec.yaml`. + +## Why + +Parallel coding agents work on several branches of the same app at once. Each branch needs its +own running, isolated copy of the app's complete stack to be tested against, on the user's own +hardware, without a shared database and without becoming reachable from the internet. + +## What Changes + +- `build.yml` pushes images and a deploy artifact also for non-default branches, tagged e.g. + `branch-` (today it builds there but pushes nothing), and deletes the tag when the branch + is deleted. Pull requests from forks never push. +- In the cluster, previews are discovered from the deploy artifact's tags. +- Each preview: its own namespace, its own secrets (generated, never copied from the app), its own + volumes, a resource quota, a limit of concurrent previews per app; deleted with its tag. +- Previews are never exposed through a tunnel in this phase; they are reached with + `shelf app open --preview `. + +## Capabilities + +### New Capabilities + +- `branch-previews` (planned). + +### Modified Capabilities + +To be decided at `/opsx:propose`; likely `app-delivery`, `app-chart` and the reusable workflow. + +## Impact + +`.github/workflows/build.yml`, `platform/apps`, `internal/cluster`, `internal/ops`, CLI. Level 2. + +## Roadmap text (from the re-cut of 2026-10-27) + +A preview gets the app's complete stack, including its own database instance, in its own +namespace. The platform knows components, not databases; opinionated providers (e.g. +`resources: db: {type: postgres}`) may come later as an optional layer. + +**To verify at propose:** the Flux Operator's input provider for OCI artifact tags (the embedded +operator is v0.60.0), its tag filtering and polling interval. Fallback: the host service maintains +one provider per preview, as long as no deploy path depends on a forge API. + +**Open:** naming of preview namespaces and hosts within the 40-character limit and without +colliding with other app names (e.g. a reserved separator, or a label-based namespace name). + +Rejected alternatives, with their reasons: + +- Flux Operator `GitHubPullRequest` / `GitHubBranch` input providers: the cluster would talk to + the GitHub API, against the guardrail that the registry is the only interface between build and + platform. The workflow pushes an artifact per branch instead. +- A shared Postgres instance for previews: cheaper, but opinionated, and against "everything is a + component" and independent environments. +- A vcluster per preview: heavier than namespaces; only worth it for apps that bring CRDs or + cluster-scoped objects. Namespaces with quotas (and the NetworkPolicies of Phase 12) are the + default. +- A local build path into the cluster for fast iteration: the devcontainer is the inner loop; + previews verify the built artefact. + +**Acceptance (level 2):** two branches run side by side, isolated, each with its own database; +deleting a branch removes its preview; RAM per preview is measured and recorded. `examples/hello` +and `examples/tenant` serve as the reference apps (formerly Phase 13). diff --git a/openspec/changes/phase-13-reference-apps/.openspec.yaml b/openspec/changes/phase-11-verification-chain/.openspec.yaml similarity index 64% rename from openspec/changes/phase-13-reference-apps/.openspec.yaml rename to openspec/changes/phase-11-verification-chain/.openspec.yaml index 9459bc2..ad9e5be 100644 --- a/openspec/changes/phase-13-reference-apps/.openspec.yaml +++ b/openspec/changes/phase-11-verification-chain/.openspec.yaml @@ -1,3 +1,3 @@ schema: spec-driven -created: 2026-10-20 +created: 2026-10-27 skip_specs: true diff --git a/openspec/changes/phase-11-verification-chain/proposal.md b/openspec/changes/phase-11-verification-chain/proposal.md new file mode 100644 index 0000000..18735ed --- /dev/null +++ b/openspec/changes/phase-11-verification-chain/proposal.md @@ -0,0 +1,54 @@ +# Proposal + +> Draft. This change holds the roadmap text only and is marked `skip_specs: true`. +> `/opsx:propose` turns it into a full change: it writes the delta specs, `design.md` and +> `tasks.md`, revises this proposal, and removes `skip_specs` from `.openspec.yaml`. + +## Why + +A preview may run anything a branch built. Before an app goes beyond a preview, shelf has to know +that the running digest is the one CI built from a given commit, signed by the expected identity +and scanned. + +## What Changes + +- `build.yml`: multi-arch build option (arm64 default for shelf hosts, amd64 for cloud targets), + image scan (e.g. Trivy), signature and build provenance for images and the deploy artifact + (cosign keyless / GitHub artifact attestations), the scan result attached as an attestation. +- Cluster: Flux verifies the deploy artifact's signature (OCIRepository `verify`); an admission + policy accepts only signed image digests from the expected identity. +- `shelf app status` shows the chain: commit → digest → signature → scan → deployed. +- Exposure gate: `--cloudflare` and `--quick` refuse an app whose running digest is not verified. + +## Capabilities + +### New Capabilities + +- `verification` (planned). + +### Modified Capabilities + +To be decided at `/opsx:propose`; likely `app-delivery`, `app-exposure`, `cluster-init` and the +reusable workflow. + +## Impact + +`.github/workflows/build.yml`, `platform/`, `internal/cluster`, `internal/ops`, CLI and admin UI. +Level 2. + +## Roadmap text (from the re-cut of 2026-10-27) + +Three separate states per app and preview: **build** (built, scanned, signed), **deployment** +(which digest runs where), **exposure** (cluster only, port-forward, tunnel). Only a verified +artefact goes beyond a preview. + +**Open:** Kyverno vs. Sigstore policy-controller for the admission policy; the RAM footprint on a +small host matters. + +This change adds the guardrail "only a verified digest can be exposed beyond a preview" to +CLAUDE.md. It was deliberately not adopted with the re-cut, because it describes behaviour that +does not exist before this phase. shelf is not marketed as more secure than alternatives before +this chain exists. + +**Acceptance (level 2):** an unsigned or unscanned artefact does not reach the cluster; exposing +an unverified app is refused; the chain is queryable per running version. diff --git a/openspec/changes/phase-11-wizard/proposal.md b/openspec/changes/phase-11-wizard/proposal.md deleted file mode 100644 index c6bec38..0000000 --- a/openspec/changes/phase-11-wizard/proposal.md +++ /dev/null @@ -1,41 +0,0 @@ -# Proposal - -> Draft. This change holds the roadmap text only and is marked `skip_specs: true`. -> `/opsx:propose` turns it into a full change: it writes the delta specs, `design.md` and -> `tasks.md`, revises this proposal, and removes `skip_specs` from `.openspec.yaml`. - -## Why - -Someone who has never seen Kubernetes should get from the one-liner to a reachable app without opening a terminal. - -## What Changes - -- A browser wizard: claim, host preflight, Colima, cluster, connections, the first app. -- Note: the roadmap text still names "the cluster's domain" as a step. Since Phase 8d a cluster has no domain; a domain belongs to an app. The proposal has to drop that step. - -## Capabilities - -### New Capabilities - -- `setup-wizard` (planned), and changes to `admin-ui`. - -### Modified Capabilities - -To be decided at `/opsx:propose`. - -## Impact - -`internal/server`, `internal/ops`. Level 2 for the UI, level 3 for the host steps. - -## Roadmap text (from the plan before OpenSpec) - -Claim → host preflight → Colima → the cluster's domain → cluster as one job → the connections (a -registry login and a Cloudflare token, each checked when it is entered) → the first app, choosing -them and its domain (the zone picked from the Cloudflare API rather than typed), with the two -files the tenant repository needs offered for copying, `` already filled in. - -**Acceptance:** someone who has never seen Kubernetes gets from the one-liner to a reachable -`https://.` without opening a terminal, given only a registry PAT, a Cloudflare token -and a tenant repository; re-running the wizard on a configured host changes nothing and shows the -current state; a wrong token is rejected on the step that collects it, naming the permission that -is missing. diff --git a/openspec/changes/phase-12-hardening/.openspec.yaml b/openspec/changes/phase-12-hardening/.openspec.yaml new file mode 100644 index 0000000..ad9e5be --- /dev/null +++ b/openspec/changes/phase-12-hardening/.openspec.yaml @@ -0,0 +1,3 @@ +schema: spec-driven +created: 2026-10-27 +skip_specs: true diff --git a/openspec/changes/phase-12-hardening/proposal.md b/openspec/changes/phase-12-hardening/proposal.md new file mode 100644 index 0000000..e75e179 --- /dev/null +++ b/openspec/changes/phase-12-hardening/proposal.md @@ -0,0 +1,45 @@ +# Proposal + +> Draft. This change holds the roadmap text only and is marked `skip_specs: true`. +> `/opsx:propose` turns it into a full change: it writes the delta specs, `design.md` and +> `tasks.md`, revises this proposal, and removes `skip_specs` from `.openspec.yaml`. + +## Why + +A preview runs code an agent wrote. It must not reach other apps or the home LAN, and a preview +should count as green only when the app's own checks pass against it. + +## What Changes + +- Pod Security labels on app and preview namespaces. +- Default-deny NetworkPolicies per app namespace: traffic only within the namespace and from + Traefik; egress to the internet allowed, egress to private address ranges outside the cluster + (the home LAN) denied. +- Smoke tests by convention: an optional test component that runs against the preview and reports + by exit code; a preview counts as green only when it passes. +- A verification report per branch: preview address, smoke test result, scan result, what changed. + +## Capabilities + +### New Capabilities + +To be decided at `/opsx:propose`; possibly `isolation` and `preview-checks`. + +### Modified Capabilities + +To be decided at `/opsx:propose`; likely `app-chart`, `app-manifest`, `branch-previews`. + +## Impact + +`charts/shelf-app`, `platform/apps`, `internal/schema`, `internal/ops`, CLI. Level 2. + +## Roadmap text (from the re-cut of 2026-10-27) + +**Open:** `restricted` breaks many stock images (e.g. Postgres starting as root); likely +`baseline` enforced plus `restricted` warned, with a per-app opt-in to `restricted`. + +**Open:** whether the dev cluster's k3d network policy controller enforces the LAN egress rule the +same way k3s on a host does, i.e. what level 2 can prove. + +**Acceptance (level 2):** a preview cannot reach another app's namespace or a LAN address; a +failing smoke test marks the preview red. diff --git a/openspec/changes/phase-10-service-installer/.openspec.yaml b/openspec/changes/phase-13-host/.openspec.yaml similarity index 100% rename from openspec/changes/phase-10-service-installer/.openspec.yaml rename to openspec/changes/phase-13-host/.openspec.yaml diff --git a/openspec/changes/phase-9-mac-mini/proposal.md b/openspec/changes/phase-13-host/proposal.md similarity index 80% rename from openspec/changes/phase-9-mac-mini/proposal.md rename to openspec/changes/phase-13-host/proposal.md index bab454b..c71d5c5 100644 --- a/openspec/changes/phase-9-mac-mini/proposal.md +++ b/openspec/changes/phase-13-host/proposal.md @@ -6,11 +6,15 @@ ## Why -Everything so far runs in the devcontainer against k3d. The appliance needs the real machine: a Mac mini that is prepared, runs Colima, and can be diagnosed and reset without a developer at the keyboard. +Everything so far runs in the devcontainer against k3d. shelf needs a real host: a Linux machine +with k3s, or a Mac that runs the same Linux in a Colima VM, prepared, diagnosable and resettable +without a developer at the keyboard. ## What Changes -- `shelf init host`: preflight, tool installation, energy settings, host name, Colima profile. +- `shelf init host`, split into "provide the environment" (macOS: preflight, tools, energy + settings, host name, Colima VM; Linux: preflight and k3s directly) and "install shelf's + components", which is the same on both. - `shelf doctor` on top of the five-stage diagnosis, and `shelf destroy [--purge]`. - The `shelf init` wrapper, `just push-mini` and the SSH access from the devcontainer. @@ -18,7 +22,7 @@ Everything so far runs in the devcontainer against k3d. The appliance needs the ### New Capabilities -- `host-setup` (planned): preflight and preparation of the Mac. +- `host-setup` (planned): preflight and preparation of the host, macOS and Linux. - `diagnostics` (planned, or a change to `app-delivery`): `shelf doctor`. - `teardown` (planned): `shelf destroy`. @@ -28,7 +32,18 @@ To be decided at `/opsx:propose`. ## Impact -`internal/host` (tested through the fake command runner only), CLI, justfile, `.devcontainer/ssh_config`. Level 3: needs the mini. +`internal/host` (tested through the fake command runner only), CLI, justfile, `.devcontainer/ssh_config`. Level 3: needs the mini and a Linux host. + +## Re-cut of 2026-10-27 + +Formerly Phase 9 (Mac mini). The core thesis (Phases 9 to 12) now comes first, and the Mac +lock-in is dropped: shelf runs on any Linux host with k3s, and on macOS the installer creates the +Colima VM and then runs the same Linux bootstrap. Tier 1: macOS (Apple Silicon, Colima) and +Ubuntu/Debian (arm64, amd64). The guardrail "no Colima, macOS or distribution assumptions outside +`internal/host`" covers both. + +**Open:** what the Linux preflight checks (k3s already installed or not, cgroup setup, distribution +versions), and whether k3s is installed by its upstream script or by shelf. ## Roadmap text (from the plan before OpenSpec) @@ -93,7 +108,7 @@ on the mini with "exec format error". `just push-mini` therefore builds explicit unknown of the new plan, because it decides whether automatic login is required: an agent in `~/Library/LaunchAgents` loads when a user session is created, and a headless mini has nobody to log in. Plan: ship automatic login as the documented path, run the experiment with - `launchctl bootstrap user/` from a daemon in Phase 9, and drop automatic login only if it + `launchctl bootstrap user/` from a daemon in this phase, and drop automatic login only if it demonstrably works without it. Automatic login in turn requires FileVault to be off, so the disk holding the tokens and the secret backups is unencrypted — accepted for a home appliance whose threat model is the LAN, but it belongs in the documentation, not in a footnote. diff --git a/openspec/changes/phase-13-reference-apps/proposal.md b/openspec/changes/phase-13-reference-apps/proposal.md deleted file mode 100644 index f8f9e68..0000000 --- a/openspec/changes/phase-13-reference-apps/proposal.md +++ /dev/null @@ -1,32 +0,0 @@ -# Proposal - -> Draft. This change holds the roadmap text only and is marked `skip_specs: true`. -> `/opsx:propose` turns it into a full change: it writes the delta specs, `design.md` and -> `tasks.md`, revises this proposal, and removes `skip_specs` from `.openspec.yaml`. - -## Why - -shelf is proven by running real apps, not only the examples. - -## What Changes - -- `examples/hello` and `examples/tenant` as reference apps, then the maintainer's own apps as the first real tenants. - -## Capabilities - -### New Capabilities - -Probably none; this phase checks the existing capabilities. - -### Modified Capabilities - -To be decided at `/opsx:propose`. - -## Impact - -`examples/`, tenant repositories. Level 3. - -## Roadmap text (from the plan before OpenSpec) - -`examples/hello` and `examples/tenant`, then the maintainer's own apps as the first real -tenants. diff --git a/openspec/changes/phase-11-wizard/.openspec.yaml b/openspec/changes/phase-14-service-installer/.openspec.yaml similarity index 100% rename from openspec/changes/phase-11-wizard/.openspec.yaml rename to openspec/changes/phase-14-service-installer/.openspec.yaml diff --git a/openspec/changes/phase-10-service-installer/proposal.md b/openspec/changes/phase-14-service-installer/proposal.md similarity index 87% rename from openspec/changes/phase-10-service-installer/proposal.md rename to openspec/changes/phase-14-service-installer/proposal.md index 1930fe2..7de3c55 100644 --- a/openspec/changes/phase-10-service-installer/proposal.md +++ b/openspec/changes/phase-14-service-installer/proposal.md @@ -26,7 +26,17 @@ To be decided at `/opsx:propose`. ## Impact -CLI, `install.sh`, `internal/host`. Level 3: needs the mini. +CLI, `install.sh`, `internal/host`. Level 3: needs the mini and a Linux host. + +## Re-cut of 2026-10-27 + +Formerly Phase 10. The installer stays one step on a Mac and also has to cover Linux hosts +(Phase 13): internally it is split into "provide the environment" and "install shelf's +components". + +**Open:** the service on Linux (a systemd user unit next to the LaunchAgent, or a system unit), +and how `install.sh` tells the two platforms apart. The setup wizard (formerly Phase 11) is no +longer the target of this phase's acceptance; the installer ends with the URL and the setup code. ## Roadmap text (from the plan before OpenSpec) diff --git a/openspec/changes/phase-12-operations/.openspec.yaml b/openspec/changes/phase-15-operations/.openspec.yaml similarity index 100% rename from openspec/changes/phase-12-operations/.openspec.yaml rename to openspec/changes/phase-15-operations/.openspec.yaml diff --git a/openspec/changes/phase-12-operations/proposal.md b/openspec/changes/phase-15-operations/proposal.md similarity index 88% rename from openspec/changes/phase-12-operations/proposal.md rename to openspec/changes/phase-15-operations/proposal.md index 667f829..8763853 100644 --- a/openspec/changes/phase-12-operations/proposal.md +++ b/openspec/changes/phase-15-operations/proposal.md @@ -28,6 +28,12 @@ To be decided at `/opsx:propose`. `internal/server`, `internal/cluster`, `internal/ops`, release artifacts. Level 2, and level 3 for the update. +## Re-cut of 2026-10-27 + +Formerly Phase 12; content unchanged. With previews (Phase 10) the logs and resource usage cover +previews as well as apps, and the disk warning matters more, since every preview brings its own +volumes. + ## Roadmap text (from the plan before OpenSpec) Live pod logs including the previous container of a crash-looping pod, instantaneous CPU and diff --git a/openspec/changes/phase-16-eject/.openspec.yaml b/openspec/changes/phase-16-eject/.openspec.yaml new file mode 100644 index 0000000..ad9e5be --- /dev/null +++ b/openspec/changes/phase-16-eject/.openspec.yaml @@ -0,0 +1,3 @@ +schema: spec-driven +created: 2026-10-27 +skip_specs: true diff --git a/openspec/changes/phase-16-eject/proposal.md b/openspec/changes/phase-16-eject/proposal.md new file mode 100644 index 0000000..1616fcf --- /dev/null +++ b/openspec/changes/phase-16-eject/proposal.md @@ -0,0 +1,60 @@ +# Proposal + +> Draft. This change holds the roadmap text only and is marked `skip_specs: true`. +> `/opsx:propose` turns it into a full change: it writes the delta specs, `design.md` and +> `tasks.md`, revises this proposal, and removes `skip_specs` from `.openspec.yaml`. + +## Why + +shelf is pre-production: it is where an app is built and verified. Production is an independent +cluster that depends only on GHCR and its own fleet repository, so that it keeps running when +shelf is down. The verified state has to get there as standard artefacts, without depending on +shelf. + +## What Changes + +- `shelf target add`: register a target cluster and run its preflight. +- `shelf eject plan --target `: render with the target's profile, show the differences, + block unverified or local artefacts. +- `shelf eject apply`: open a pull request in the target's fleet repository with plain + Flux/Helm/policy objects and the same verified digest. Later versions arrive as pull requests + that bump the digest (promotion), or the repository continues on its own (eject). + +## Capabilities + +### New Capabilities + +- `targets`, `eject` (planned). + +### Modified Capabilities + +To be decided at `/opsx:propose`. + +## Impact + +CLI, `internal/ops`, a new package for target profiles and fleet repositories. Level 3: needs a +real target cluster. + +## Roadmap text (from the re-cut of 2026-10-27) + +First reference target: the maintainer's Hetzner k3s/Flux setup. + +Needed from the start, and to be respected by earlier phases: target profiles (the shelf host is +just the first target), a portable secrets format (e.g. SOPS per target, or External Secrets), +per-target component overrides (replace a component with env pointing at an external service), +artefacts always in GHCR. The guardrail "the runtime output depends on no shelf-specific CRD" was +adopted with the re-cut for this phase. + +**Open:** opening a pull request is a forge API call from the host. The guardrail allows the host +service to call the GitHub API as long as no deploy path depends on it; whether a path into +production counts as a deploy path has to be decided here. + +shelf cannot test high availability, provision cloud infrastructure or migrate data automatically; +it can guide a data migration and a DNS cut-over. + +Later, building on the target preflight: `shelf init --existing-cluster`, which installs only +shelf's components into an existing k3s/Kubernetes cluster. + +**Acceptance (level 3):** a verified app from the shelf host runs on the Hetzner target from a +merged pull request with the same digest, and keeps running with shelf switched off; the +maintainer's own apps are the first real tenants (formerly Phase 13). diff --git a/openspec/changes/phase-9-agent-interface/.openspec.yaml b/openspec/changes/phase-9-agent-interface/.openspec.yaml new file mode 100644 index 0000000..97e3642 --- /dev/null +++ b/openspec/changes/phase-9-agent-interface/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-27 diff --git a/openspec/changes/phase-9-agent-interface/design.md b/openspec/changes/phase-9-agent-interface/design.md new file mode 100644 index 0000000..0cc28ff --- /dev/null +++ b/openspec/changes/phase-9-agent-interface/design.md @@ -0,0 +1,239 @@ +# Design + +## Context + +See proposal.md for the motivation. The facts the approach rests on: + +- `ops.App`, `cluster.AppState`, `cluster.Diagnosis` and `cluster.Stage` already carry `json` + tags, because the admin UI's `/api/` serves them. That API is explicitly unstable ("`/api/` + serves the UI and may change", context), so it is not the contract for agents. +- Flux copies the deploy artifact's OCI annotations into `status.artifact.metadata` of the + OCIRepository. On the dev cluster on 2026-10-27, the app `greeter` shows + `org.opencontainers.image.revision: main@sha1:42a964eb…` and + `org.opencontainers.image.source: https://github.com/tweinmann/shelf-hello`. The reusable + workflow sets them with `flux push artifact --revision "$GITHUB_REF_NAME@sha1:$GITHUB_SHA"`. + So the cluster knows the commit without any forge API. +- `shelf app add` avoids the stale-status race by asking each controller to reconcile and + waiting for the request to be handled (`rollOut`, `reconcileAndWait`). `--wait` is read-only + (decision 4), so it needs a different way to tell "Ready for the new artifact" from "still Ready + from the old one". +- The HelmRelease waits for its resources (Helm's wait is on by default and the platform does + not turn it off), so a Ready release means the Deployments and StatefulSets have rolled out. + `shelf-values` carries `reconcile.fluxcd.io/watch: Enabled`, so helm-controller reacts to a new + ConfigMap at once. +- The deploy artifact is polled every minute (`interval: 1m` in the ResourceSet). +- Traefik is the ClusterIP Service `traefik/traefik`, port 80 → entry point `web`. cloudflared + of every tunnel talks to it at `traefik.traefik.svc.cluster.local:80` and rewrites the Host for + quick tunnels (`--http-host-header`). +- `cli.Execute` maps every error to exit status 1; `ErrReported` suppresses the second print. + +## Goals / Non-Goals + +**Goals:** +- A JSON contract an agent can rely on across releases, tested by golden files. +- A wait that gives a true answer for one specific commit, without writing to the cluster. +- Browser access to any app on the developer's machine, with routing identical to a tunnel. + +**Non-Goals:** +- JSON for the mutating commands (`add`, `credentials`, `redeploy`, `rm`) or a JSON event stream + of `progress.Event`s. Possible later as a new kind. +- A stable HTTP API. `/api/` stays the UI's own. +- Waiting for branch artifacts. Only `main` is pushed until Phase 10, so `--commit` of a branch + commit times out. Phase 10 adds `--preview `. +- `shelf app open` for previews (Phase 10), and any access beyond the loopback interface. +- A Claude Code hook in the tenant template. + +## Decisions + +### 1. The JSON documents are own types in `internal/cli`, versioned as `shelf.dev/v1alpha1` + +Every `--json` output is one document type with `apiVersion` and `kind` +(`AppList`, `AppStatus`, `WaitResult`, `TagList`, `ConnectionList`, `Version`, `Error`), built +from the `ops` values by explicit mapping functions in `internal/cli/json.go`. Each kind has a +golden file in `internal/cli/testdata/json/`, so a rename in `ops` or `cluster` that would change +the contract fails a test instead of reaching an agent. Field names are lowerCamelCase, times are +RFC 3339 in UTC, empty values are omitted (`omitempty`/`omitzero`), and no field is ever `null`. +An `AppStatus` nests its diagnosis as `stages`, the five stage objects in order. + +The API group is the same as app.yaml's (`shelf.dev/v1alpha1`) on purpose: it is the convention +of the project (CLAUDE.md, Conventions), and the `kind` keeps the two apart. + +*Alternatives:* marshal `ops.App` and `cluster.Diagnosis` directly (cheapest, but every internal +rename becomes a breaking change for agents, and the UI's needs leak into the contract; rejected +by the maintainer). A separate package such as `internal/agentapi` (more ceremony for six types +that only the CLI produces; can be split out if the server ever serves them). An unversioned +format (no way to make an incompatible change later). + +### 2. Errors are documents too + +With `--json`, `cli.Execute` writes an `Error` document to stdout and the plain message to +stderr. An agent then reads one place, and a human watching the terminal still sees the error. +The exit status stays 1. Cobra's own usage errors (unknown flag) happen before the command runs +and stay plain text, which is acceptable: they are a bug in the caller, not a state of the +cluster. + +### 3. The commit comes from the artifact's revision annotation + +`cluster.AppState` gains `Commit`, parsed from +`status.artifact.metadata["org.opencontainers.image.revision"]` of the OCIRepository `deploy`. The +value has the form `@sha1:`, and a bare 40-digit SHA is accepted too. Anything else +leaves `Commit` empty. The text output of `shelf app status ` gains a `commit` line, and the +JSON documents carry `commit`. This reads the cluster only, so the registry stays the only +interface between build and platform. + +*Alternatives:* read the commit from the image label `org.opencontainers.image.revision` (needs a +registry call per image, and a component with `image:` has no label of ours). Ask the GitHub API +(against the guardrail). + +### 4. `--wait` is a read-only poll with a "caught up" check + +`ops.Wait(ctx, app, commit, timeout)` polls every 2 seconds (the existing `pollInterval`) with a +new `cluster.AppRollout` read: the diagnosis plus what the stages have caught up with. In order: + +1. **Commit.** With `--commit`, the deploy artifact's `Commit` must have the given prefix. Until + it has, the verdict-to-be is Working at "the deploy artifact" ("waiting for commit 42a964e; + running 9956993"). Only a Failed deploy artifact stage ends the wait early here (e.g. the + registry refuses the login, so the new artifact can never arrive). Failed later stages belong + to the previous artifact and are ignored. +2. **App values.** The Kustomization's `status.lastAppliedRevision` must equal the OCIRepository's + `status.artifact.revision`, as `rollOut` already checks with `appliedRevision`. A Failed + values stage counts only when `status.lastAttemptedRevision` is that revision too; otherwise + it is the previous artifact's failure. +3. **Release.** shelf computes the digest of the values the release has to run: the ConfigMap + `shelf-values` (key `app.yaml`) with the HelmRelease's inline `spec.values` merged on top, + encoded as YAML with `sigs.k8s.io/yaml` and hashed with SHA-256. That is helm-controller's + `configDigest`, reproduced byte for byte against the dev cluster (see "Spike" below), and a + unit test pins it to a digest helm-controller wrote. Then: + - The release *is about the current values* when `status.history[0].configDigest` equals that + digest and `Reconciling` is not True. Only then do its Ready or Failed count. + - An attempt *in flight is for the current values* when `status.lastAttemptedConfigDigest` + equals it. Only then may a Failed pod stage end the wait early, because the release waits for + its pods and would otherwise report the failure only after Helm's 5-minute timeout. + - Anything else is Working ("waiting for the release to pick up the new values"). This covers + an upgrade of the previous values that is still running, the rollback after it, and the + retry. +4. **Diagnosis.** After that, the five stages decide: all Ready means `Ready`, and a Failed stage + means `Failed` with that stage. + +**Spike (2026-10-27, dev cluster).** A broken image was pushed, and the fix was pushed while +Helm was still waiting for the broken upgrade. What the HelmRelease reported: + +| Time | HelmRelease | +|---|---| +| 21:13:15 | upgrade starts: `Reconciling`/`Ready=Unknown` "Running 'upgrade'", `lastAttemptedConfigDigest` = digest of the broken values, `history` unchanged | +| 21:18:07 | the fix's values are applied by kustomize-controller | +| 21:18:16 | broken upgrade times out: `Ready=False UpgradeFailed`, history v2 `failed`, `lastDeployed` 21:13:16 (the attempt's start) | +| 21:18:18 | remediation rollback: history v3 `deployed`, `lastDeployed` 21:18:18 (after the fix's values), digest of the old values; `Reconciling` True | +| 21:18:35 | upgrade to the fix: `Ready=True UpgradeSucceeded`; no new revision, because v3 already has those values | + +The first version of this decision compared timestamps: the release's latest history entry or +Ready transition against the values' last change. `smoke-agent` showed it failing. The broken +upgrade's `UpgradeFailed` at 21:18:16 is newer than the fix's values, so the fix got the +verdict Failed. Comparing only the start of the latest attempt does not help either: the +rollback's entry is newer than the fix's values and carries the old ones. Only the digest says +which values a release holds. + +Without `--commit`, steps 2 to 4 still apply, so a wait that starts in the middle of a rollout +does not report the old release as Ready. + +The verdict (`WaitResult`) holds `outcome`, `app`, `commit`, `revision`, and for anything but +Ready the `stage` with `name`, `phase`, `reason`, `message` and `hint`. The text form is three +lines at most (outcome, stage and reason, hint). The default timeout is 10 minutes, which covers +a CI build of a few minutes plus the one-minute poll. + +The exit statuses are 0, 1 and 2. They travel as a new `cli.ExitError{Code}` that +`cli.Execute` maps, after printing the verdict, without a second error line. + +*Alternatives:* ask Flux to reconcile like `rollOut` does (faster by up to a minute, but status +would become a write that has to be audit-logged; rejected by the maintainer). Compare timestamps +of the values and the release (the first version of this decision; wrong after a failed upgrade, +see the spike). Watch instead of poll (more code for a loop that runs once per agent push). + +### 5. `shelf app open`: client-go port-forward to a Traefik pod, plus a local reverse proxy + +`internal/cluster` gets `ForwardTraefik(ctx, cfg)`. It resolves the Service `traefik/traefik`, +picks a ready pod from its EndpointSlices, maps the service port 80 to the pod's `web` port, and +opens a client-go `tools/portforward` session on `127.0.0.1:0`. It returns that local address +and a channel that closes when the session ends. This talks to the API server only: no kubectl, +and nothing about HTTP. + +`internal/ops` builds the proxy: an `httputil.ReverseProxy` to the forwarded address whose +`Rewrite` sets `Host` to `AppHost(app, suffix, domain)`, the app's own host and never the quick +tunnel's random name. It also sets `X-Forwarded-Host` and `X-Forwarded-Proto: http` to the local +address. A `Location` header pointing at the app's own host is rewritten to the local address, +so redirects stay in the browser's reach. Upgrades (WebSocket) pass through, because +`ReverseProxy` supports them. When the session ends (Traefik pod restarted), the next request +opens a new one. Requests in flight fail with 502. + +`internal/cli` binds `127.0.0.1:` itself, with no flag for the address (spec "Local access +stays on the loopback interface"), prints the URL and serves until the context ends. +`--browser` calls `Options.OpenBrowser`. That seam defaults to `open` on macOS and `xdg-open` on +Linux, and tests replace it. + +*Alternatives:* the API server's service proxy (`/api/v1/namespaces/traefik/services/…/proxy/`), +which `quickURL` uses for one GET. It rewrites paths and links, mangles redirects and does not +carry WebSockets well, so routing would no longer be identical to a tunnel. Forwarding straight +to the app's pod bypasses Traefik, `stripPrefix` and the multi-component routing. A raw TCP +forward without a proxy cannot set the Host header, so Traefik would not route. + +### 6. Tenant template: a skill and a `CLAUDE.md` section, no hook + +`examples/tenant/.claude/skills/shelf/SKILL.md` describes the loop: +1. Commit and push. +2. Run `shelf app status --wait --commit "$(git rev-parse HEAD)" --json`. +3. On exit status 1, read `stage`, `reason` and `hint`, fix, and push again. On exit status 2, + check CI first. + +It also says what an agent cannot do: exposure is the operator's decision, and `shelf app open` +is for the person at the machine. The `CLAUDE.md` section names the app (``, filled in by +the operator) and points at the skill. A PostToolUse hook was rejected: a wait of several minutes +would hit the hook timeout and block the agent's turn. + +### 7. Level-2 acceptance in a new `just smoke-agent` + +The new smoke test does the following: +- It pushes `examples/hello` to the dev registry with `--revision main@sha1:` (as + `smoke-apps` already does). +- It adds the app and checks `status --json` with `jq`. +- It starts `--wait --commit ` before pushing a second artifact, and expects exit status 0 + only after the new commit's release is Ready. +- It pushes an artifact whose image cannot be pulled, and expects exit status 1 with the stage + "the pods". +- It checks `--timeout 5s --commit `, which must give exit status 2. +- It runs `shelf app open` against a fixture app with a `/api` route with `stripPrefix` + (`traefik/whoami` echoes the path it got), and checks both routes and the Host Traefik saw + with `curl`. +- It removes its app. + +It runs in CI with the other level-2 smoke tests. + +## Risks / Trade-offs + +- **The values digest reproduces helm-controller's.** If a Flux release changes how + helm-controller merges or hashes values, no release ever matches, and every wait ends TimedOut + at "the app" instead of giving a wrong verdict. → shelf pins Flux through its platform release; + the unit test holds a digest helm-controller wrote, and `smoke-agent` fails on the first wait if + the digests disagree. Both run before a Flux update ships. +- **A fix after a broken push waits for Helm.** helm-controller finishes (times out) the broken + upgrade and its rollback before it starts the next one, which can take Helm's 5-minute timeout. + → The verdict says "waiting for the release to pick up the new values" meanwhile, and the + tenant skill tells the agent to expect it; `--timeout` defaults to 10 minutes. +- **A Failed pod stage can be transient** (an image pull that would succeed on a retry). → Report + it as the verdict anyway. Kubernetes' own back-off makes "retry later" the agent's call, and + the hint says what failed. +- **The commit annotation is a convention of `flux push artifact`.** An artifact pushed some + other way has none, and `--commit` then times out. → The verdict says "the artifact names no + commit" instead of only "timed out". +- **The port-forward needs `pods/portforward` in `traefik`.** The cluster-admin kubeconfig shelf + uses has it. A narrower kubeconfig gets the API server's 403, which the command passes on. +- **Absolute URLs an app builds from the Host header point at `.shelf.internal`.** Quick + tunnels have the same issue. → Rewrite `Location` headers (decision 5). Links inside page + bodies stay the app's responsibility, as they do behind any proxy. +- **The JSON contract can't be made stable after the fact.** → Golden files per kind from the + first release, and the rule in the spec "Stable fields within a version". + +## Migration Plan + +No migration. The new fields and flags are additive. The text output of `shelf app status ` +gains a `commit` line, and its golden files are rewritten with `just golden`. Rollback is the +previous release. The platform artifact and the chart do not change. diff --git a/openspec/changes/phase-9-agent-interface/proposal.md b/openspec/changes/phase-9-agent-interface/proposal.md new file mode 100644 index 0000000..6dbce4a --- /dev/null +++ b/openspec/changes/phase-9-agent-interface/proposal.md @@ -0,0 +1,60 @@ +# Proposal + +## Why + +shelf's users run several coding agents in parallel, and each agent has to learn whether its +push arrived and what is broken. Today it has to parse text written for people, and it cannot +tell its new commit from the old one that was already running. The person behind the agents +cannot open a private app in a browser without giving it a public address. Both block the core +thesis: branch previews (Phase 10) are only useful if an agent can wait for one and read its +verdict, and if a person can look at it without exposing it. + +## What Changes + +- `--json` on the read-only agent-facing commands: `shelf app status` (all apps and one app), + `shelf app tags`, `shelf connection list` and `shelf version`. The output is a versioned + document with its own types (`apiVersion: shelf.dev/v1alpha1`, a `kind`), changed only + additively within a version. `shelf app secrets` gets no `--json`, so secret values don't end up + in an agent's transcript, and the mutating commands keep their text output. +- `shelf app status --wait [--commit ] [--timeout]`: waits until the app's chain + is Ready or Failed. With `--commit`, it first waits until the deploy artifact carries that git + commit. It prints a condensed verdict (outcome, stage, reason, hint) and exits 0 (Ready), 1 + (Failed) or 2 (timed out). It stays read-only: it never asks Flux to reconcile. +- `shelf app status` shows the git commit the running deploy artifact was built from. shelf reads + it from the artifact's revision annotation, which the OCIRepository reports, so no forge API is + involved. +- `shelf app open [--port] [--browser]`: a local listener on `127.0.0.1` only that forwards + through the API server to Traefik and sets the Host header to the app's own host, so routes and + `stripPrefix` behave as they do through a tunnel. It runs until it is stopped. CLI only. +- `examples/tenant` gains a Claude Code skill (`.claude/skills/shelf/SKILL.md`) and a `CLAUDE.md` + section that explain the loop: push, wait for the commit, read the verdict, fix. There is no + hook. + +## Capabilities + +### New Capabilities + +- `agent-interface`: the machine-readable contract for agents: the versioned JSON documents and + which commands offer them, the wait verdict with its exit codes, and the agent instructions in + the tenant template. + +### Modified Capabilities + +- `app-delivery`: the app status command also shows the commit the running artifact was built + from. +- `app-exposure`: a new way to reach an app: a local port-forward to Traefik with the app's Host, + for private and exposed apps alike, on the loopback interface only. + +## Impact + +- `internal/cluster`: reads the commit from the OCIRepository status, and opens a port-forward + connection to a Traefik pod through the API server (client-go `tools/portforward`, no kubectl). +- `internal/ops`: the wait loop, and the Host-rewriting proxy for `open`. +- `internal/cli`: the JSON document types, `--json`, `--wait`, `--commit` and the `open` command. + Golden files for every JSON document. +- `examples/tenant`: skill and `CLAUDE.md`. README CLI reference. +- No new direct Go dependency. client-go's port-forward brings three indirect modules of its own + graph (`gorilla/websocket`, `moby/spdystream`, `k8s.io/streaming`). No change to the platform + artifact, the chart or `build.yml`. +- Level 1 for the JSON documents and the verdict logic (fake cluster). Level 2 for `--wait` + against a real rollout and for `open`, in a new `just smoke-agent`. diff --git a/openspec/changes/phase-9-agent-interface/specs/agent-interface/spec.md b/openspec/changes/phase-9-agent-interface/specs/agent-interface/spec.md new file mode 100644 index 0000000..570207b --- /dev/null +++ b/openspec/changes/phase-9-agent-interface/specs/agent-interface/spec.md @@ -0,0 +1,146 @@ +# Spec Delta + +## Purpose + +The contract between shelf and the coding agents that work in tenant repositories: machine-readable +JSON documents for the read-only commands, a wait that ends in a verdict an agent can act on, and +the instructions the tenant template gives an agent about that loop. + +## ADDED Requirements + +### Requirement: Versioned JSON documents +A command run with `--json` SHALL print exactly one JSON object to stdout and nothing else there, +with the fields `apiVersion` (`shelf.dev/v1alpha1`) and `kind` naming the document. Progress, +hints and warnings SHALL go to stderr only. + +#### Scenario: Status of all apps as JSON +- **WHEN** a user runs `shelf app status --json` against a cluster with the apps `greeter` and `hello` +- **THEN** stdout is one JSON object with `apiVersion` `shelf.dev/v1alpha1`, `kind` `AppList` and an `items` array with one entry per app +- **AND** stdout parses as JSON without stripping any line + +#### Scenario: No apps +- **WHEN** a user runs `shelf app status --json` against a cluster without apps +- **THEN** it prints an `AppList` with an empty `items` array instead of "No apps yet." + +### Requirement: Errors as JSON +A command run with `--json` that fails SHALL print a document of kind `Error` with a `message` to +stdout, write the same message to stderr, and exit with status 1. + +#### Scenario: Unknown app +- **WHEN** a user runs `shelf app status nope --json` and no app `nope` exists +- **THEN** stdout is `{"apiVersion":"shelf.dev/v1alpha1","kind":"Error","message":"app nope does not exist"}` and the exit status is 1 + +#### Scenario: Cluster unreachable +- **WHEN** a user runs `shelf app status --json` and the API server does not answer +- **THEN** stdout is an `Error` document with the reason, and the exit status is 1 + +### Requirement: Stable fields within a version +Within one `apiVersion`, a field of a JSON document SHALL never be removed, renamed or change its +type or meaning; new fields MAY be added, and a consumer is expected to ignore fields it does not +know. A field without a value SHALL be omitted, never set to `null`. + +#### Scenario: Field added in a later release +- **WHEN** a later release adds a field to `AppStatus` +- **THEN** `apiVersion` stays `shelf.dev/v1alpha1` and every earlier field keeps its name, type and meaning + +#### Scenario: Incompatible change +- **WHEN** a field has to change its meaning +- **THEN** the change ships as a new `apiVersion`, and the golden file of the old document fails until it is renamed with it + +### Requirement: Commands with JSON output +`--json` SHALL be offered by `shelf app status` (kind `AppList` without a name, `AppStatus` with +one, `WaitResult` with `--wait`), `shelf app tags` (`TagList`), `shelf connection list` +(`ConnectionList`) and `shelf version` (`Version`). + +#### Scenario: One app in detail +- **WHEN** a user runs `shelf app status hello --json` for a healthy app +- **THEN** the `AppStatus` holds the app's name, phase, address, artifact, registry connection, exposure, revision, commit, deploy time, components with their addresses, and the five stages each with name, object, phase and, where present, reason, message and hint + +#### Scenario: Tags +- **WHEN** a user runs `shelf app tags hello --json` +- **THEN** the `TagList` lists the tags newest first, and the tag the app runs with has `current: true` + +#### Scenario: Version +- **WHEN** a user runs `shelf version --json` +- **THEN** the `Version` document holds the shelf version and the app.yaml `apiVersion` it reads + +### Requirement: No secret values in JSON +No JSON document SHALL contain a secret value or a token. `shelf app secrets` SHALL NOT offer +`--json`, and `ConnectionList` SHALL hold each connection's kind, name, account or user, and the +apps that use it, never its token. + +#### Scenario: Connections as JSON +- **WHEN** a user runs `shelf connection list --json` with a registry and a Cloudflare connection +- **THEN** each entry has `kind`, `name` and `apps`, the registry entry its user, the Cloudflare entry its account or that this machine does not hold it, and no field of the document holds either token + +#### Scenario: Secrets +- **WHEN** a user runs `shelf app secrets hello --json` +- **THEN** the command fails because the flag is unknown + +### Requirement: Waiting for an app to settle +`shelf app status --wait` SHALL poll the app's chain until it is Ready or a stage is +Failed, or until `--timeout` (default 10 minutes) runs out, and then print a verdict. + +#### Scenario: App already healthy +- **WHEN** a user runs `shelf app status hello --wait` for a Ready app +- **THEN** it returns at once with the outcome `Ready` + +#### Scenario: Rollout still running +- **WHEN** the app's pods are starting and become ready a minute later +- **THEN** the command returns after they are ready, with the outcome `Ready` + +### Requirement: Waiting for a commit +With `--commit `, the wait SHALL first wait until the app's deploy artifact was built from that +git commit, matching a full SHA or a prefix of at least 7 hex digits, and then until every stage +has caught up with that artifact; it SHALL NOT report Ready for the previous artifact. + +#### Scenario: Push still in CI +- **WHEN** an agent runs `shelf app status hello --wait --commit 42a964e --json` right after `git push`, and the artifact of that commit arrives four minutes later +- **THEN** the command keeps waiting while the previous artifact runs, and returns `Ready` only once the release built from `42a964e` has rolled out + +#### Scenario: Short prefix +- **WHEN** `--commit` is given fewer than 7 hex digits or something that is not hex +- **THEN** the command fails at once without waiting + +#### Scenario: Artifact without a commit +- **WHEN** the app's deploy artifact carries no git revision, as one pushed by hand may not +- **THEN** the wait for a commit runs into its timeout, and the verdict says that the artifact names no commit + +### Requirement: Wait verdict and exit status +The wait SHALL end with a verdict: outcome `Ready`, `Failed` or `TimedOut`, the commit and revision +that run, and, unless Ready, the first stage that is not Ready with its reason, message and hint. +The exit status SHALL be 0 for Ready, 1 for Failed and 2 for TimedOut. + +#### Scenario: Image cannot be pulled +- **WHEN** the awaited commit's release starts a pod that waits in `ImagePullBackOff` +- **THEN** the verdict is `Failed` with the stage "the pods", the reason `ImagePullBackOff` and the hint that the image cannot be pulled, and the exit status is 1 + +#### Scenario: Timeout +- **WHEN** the app is still Working when `--timeout 2m` runs out +- **THEN** the verdict is `TimedOut` naming the stage that is still Working, and the exit status is 2 + +#### Scenario: Error before a verdict +- **WHEN** the app does not exist or the cluster cannot be reached +- **THEN** the command prints an error, not a verdict, and exits with status 1 + +### Requirement: Waiting is read-only +`shelf app status` with or without `--wait` SHALL NOT write to the cluster, ask a Flux controller +to reconcile, or write to `~/.shelf`. + +#### Scenario: Waiting for a new artifact +- **WHEN** an agent waits for a commit that is not yet in the cluster +- **THEN** shelf only reads, and the artifact arrives at the deploy artifact's own polling interval + +### Requirement: Agent instructions in the tenant template +`examples/tenant` SHALL hold a Claude Code skill `.claude/skills/shelf/SKILL.md` and a `CLAUDE.md` +section that tell an agent to push, run `shelf app status --wait --commit --json`, read +the verdict, and fix what its stage and hint name; the app name is a placeholder the operator +fills in. + +#### Scenario: Agent loop described +- **WHEN** an agent working in a copy of `examples/tenant` reads the skill +- **THEN** it finds the wait command with `--commit` and `--json`, the meaning of the three exit statuses, and that it cannot make the app public + +#### Scenario: No hook +- **WHEN** the tenant template is inspected +- **THEN** it ships no Claude Code hook that runs the wait diff --git a/openspec/changes/phase-9-agent-interface/specs/app-delivery/spec.md b/openspec/changes/phase-9-agent-interface/specs/app-delivery/spec.md new file mode 100644 index 0000000..bba3f6e --- /dev/null +++ b/openspec/changes/phase-9-agent-interface/specs/app-delivery/spec.md @@ -0,0 +1,25 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: App status command +`shelf app status` SHALL list every app with its state and, when not Ready, the failing stage and +reason in one line; `shelf app status ` SHALL print the app's address, artifact, registry +connection, running revision, the git commit that revision was built from, deploy time, +components, and every stage with its message and, if not Ready, its hint. + +#### Scenario: No apps +- **WHEN** the cluster has no registered apps +- **THEN** `shelf app status` prints "No apps yet." + +#### Scenario: One app in detail +- **WHEN** a user runs `shelf app status hello` for a healthy app +- **THEN** the output shows `hello: Ready`, the artifact reference, the revision, and all five stages as Ready + +#### Scenario: Commit of the running artifact +- **WHEN** the app's deploy artifact was pushed by the reusable workflow for commit `42a964eb0b4b…` +- **THEN** `shelf app status hello` shows that commit, read from the artifact's revision annotation as the cluster reports it, without calling a forge API + +#### Scenario: Artifact without a commit +- **WHEN** the deploy artifact carries no git revision +- **THEN** the commit line is left out diff --git a/openspec/changes/phase-9-agent-interface/specs/app-exposure/spec.md b/openspec/changes/phase-9-agent-interface/specs/app-exposure/spec.md new file mode 100644 index 0000000..a46a49f --- /dev/null +++ b/openspec/changes/phase-9-agent-interface/specs/app-exposure/spec.md @@ -0,0 +1,46 @@ +# Spec Delta + +## ADDED Requirements + +### Requirement: Local access through a port-forward +`shelf app open ` SHALL listen on a local port and forward every request through the API +server to Traefik with the Host header set to the app's own host, so that routes and `stripPrefix` +behave as through a tunnel; it SHALL work for private and exposed apps alike and run until stopped. + +#### Scenario: Private app in a browser +- **WHEN** a user runs `shelf app open hello` for an app without exposure that routes `/` and `/api` with `stripPrefix` +- **THEN** it prints `http://127.0.0.1:/`, a request to `/api/x` reaches the `/api` component as `/x`, and no tunnel, DNS record or cloudflared is created + +#### Scenario: Stopped +- **WHEN** the user presses Ctrl-C +- **THEN** the listener and the forward are closed and the command exits with status 0 + +#### Scenario: App does not exist +- **WHEN** a user runs `shelf app open nope` +- **THEN** it fails saying that app nope does not exist, before it listens on any port + +### Requirement: Local access stays on the loopback interface +`shelf app open` MUST listen on `127.0.0.1` only and offer no option for another address. The +port SHALL be chosen freely unless `--port` names one, and `--browser` SHALL open the printed +address in the default browser; without it no browser is started. + +#### Scenario: Chosen port +- **WHEN** a user runs `shelf app open hello --port 8081` and the port is free +- **THEN** it listens on `127.0.0.1:8081` and on no other address + +#### Scenario: Port in use +- **WHEN** the port given with `--port` is in use +- **THEN** the command fails saying so, without forwarding anything + +#### Scenario: No browser by default +- **WHEN** a user runs `shelf app open hello` without `--browser` +- **THEN** no browser is started and only the address is printed + +### Requirement: Local access is not an exposure +`shelf app open` SHALL NOT change the app's registration, exposure, tunnel or DNS record, SHALL NOT +write to the cluster beyond the port-forward the API server opens, and SHALL NOT appear in the +admin UI. + +#### Scenario: Status while forwarding +- **WHEN** `shelf app open hello` runs and the user runs `shelf app status hello` +- **THEN** the app's exposure and address are unchanged diff --git a/openspec/changes/phase-9-agent-interface/tasks.md b/openspec/changes/phase-9-agent-interface/tasks.md new file mode 100644 index 0000000..e08bfde --- /dev/null +++ b/openspec/changes/phase-9-agent-interface/tasks.md @@ -0,0 +1,45 @@ +# Tasks + +## 1. Commit of the running artifact + +- [x] 1.1 Add `Commit` to `cluster.AppState`, parsed from `status.artifact.metadata["org.opencontainers.image.revision"]` of the OCIRepository `deploy` (`@sha1:` or a bare 40-digit SHA, otherwise empty); verify with table-driven cases in `internal/cluster` (workflow form, bare SHA, `local`, missing metadata) +- [x] 1.2 Print a `commit` line in `shelf app status ` (left out when empty) and update the README CLI reference; verify with `just golden` and a CLI test for both cases (spec app-delivery "Commit of the running artifact", "Artifact without a commit") + +## 2. JSON documents + +- [x] 2.1 Add `--json` plumbing to `internal/cli`: document envelope (`apiVersion`, `kind`), stdout-only output, `Error` document from `cli.Execute` with the message also on stderr; verify with a CLI test that an unknown app yields the exact `Error` JSON and exit status 1 +- [x] 2.2 Add the document types and mapping functions for `AppList` and `AppStatus` in `internal/cli/json.go`, with `shelf app status [app] --json`; verify with golden files in `internal/cli/testdata/json/` for an empty list, two apps, a healthy app and a failing app, and a test that no field is `null` +- [x] 2.3 Add `TagList` (`shelf app tags --json`, `current: true` on the running tag), `ConnectionList` (`shelf connection list --json`, no token, account or "not on this machine") and `Version` (`shelf version --json`); verify with golden files, using only obviously fake account IDs +- [x] 2.4 Check that `shelf app secrets --json` fails as an unknown flag; verify with a CLI test +- [x] 2.5 Document `--json`, the kinds and the stability rule in the README CLI reference; verify that the documented examples match the golden files + +## 3. Waiting for a verdict + +- [x] 3.1 Add `cluster.AppRollout`: the diagnosis plus the deploy artifact's revision and commit, the Kustomization's `lastAppliedRevision` and `lastAttemptedRevision`, the digest of the current values (ConfigMap `shelf-values` merged with the HelmRelease's inline values, as helm-controller computes `configDigest`), the HelmRelease's `history[0].configDigest`, `lastAttemptedConfigDigest` and whether it is `Reconciling`; add it to the `ops.Cluster` interface and the fake; verify with tests against unstructured fixtures and a digest test against the values and `configDigest` helm-controller wrote on the dev cluster (`testdata/configdigest`) +- [x] 3.2 Implement `ops.Wait(ctx, app, commit, timeout)` with the caught-up rules of design decision 4 (commit prefix, applied revision, Failed values only for an attempted revision, release digest equal and not reconciling, pod look-ahead only for an attempt of the current values, then the five stages; only a Failed deploy-artifact stage ends a commit wait early); verify with table-driven tests for: already Ready, rollout in progress, previous values still released, previous upgrade still running or rolling back, old artifact Failed while waiting for a new commit, registry refusal, image pull failure, timeout, artifact without commit +- [x] 3.3 Add `--wait`, `--commit` (at least 7 hex digits, validated before polling) and `--timeout` (default 10m) to `shelf app status `, the `WaitResult` document and its three-line text form, and `cli.ExitError` mapped by `cli.Execute` to 0/1/2 without a second error line; verify with CLI tests for each outcome and exit status, and golden files for `WaitResult` +- [x] 3.4 Verify that the wait never writes: a fake cluster that fails every write method is used in the 3.2 tests and stays untouched (spec "Waiting is read-only") +- [x] 3.5 Document `--wait`, `--commit` and the exit statuses in the README CLI reference + +## 4. `shelf app open` + +- [x] 4.1 Add `cluster.ForwardTraefik`: resolve Service `traefik/traefik`, pick a ready pod from its EndpointSlices, map port 80 to the pod's `web` port, open a client-go port-forward on `127.0.0.1:0`, return the address and a done channel; verify the pod and port selection with unit tests (no ready pod, named and numeric target ports) +- [x] 4.2 Add the proxy in `internal/ops`: `Host` set to `AppHost` (also for quick-tunnel apps), `X-Forwarded-Host`/`-Proto`, `Location` rewrite for the app's own host, reconnect after the session ends, 502 while it is down; verify with `httptest` against a fake forward (Host seen, redirect rewritten, WebSocket upgrade passes, reconnect after close) +- [x] 4.3 Add `shelf app open [--port] [--browser]`: app existence checked before listening, bind `127.0.0.1` only, clear error for a port in use, `Options.OpenBrowser` seam (`open` on macOS, `xdg-open` on Linux); verify with CLI tests (unknown app, port in use, browser called only with `--browser`, listener address is loopback) +- [x] 4.4 Document `shelf app open` in the README (CLI reference and "Deploying an app": how to reach a private app), and note in `--help` that it is not an exposure + +## 5. Tenant template + +- [x] 5.1 Add `examples/tenant/.claude/skills/shelf/SKILL.md` (push, `--wait --commit "$(git rev-parse HEAD)" --json`, the three exit statuses, read stage/reason/hint, no exposure, `shelf app open` is for the person) and a `CLAUDE.md` section with the `` placeholder; verify that `examples/tenant` contains no `.claude/settings.json` hook and that the commands in the skill match the CLI's `--help` +- [x] 5.2 Mention the agent loop in `examples/tenant/README.md` and in the README section "Deploying an app" + +## 6. Level-2 acceptance + +- [ ] 6.1 Add `hack/smoke/agent.sh` and `just smoke-agent` as in design decision 7 (status JSON, wait for a new commit with exit 0 only after it rolled out, failing image with exit 1, unknown commit with exit 2, `open` against a whoami fixture with `/api` + `stripPrefix`, cleanup); verify that `just smoke-agent` passes on a fresh `just cluster-reset` +- [ ] 6.2 Add `smoke-agent` to the level-2 matrix in `.github/workflows/ci.yml` and to the CLAUDE.md dev-environment table; verify that the CI job runs it on a pull request +- [x] 6.3 Run `just test` and `openspec validate --all --strict`; both pass + +## Workflow follow-up + +- Acceptance by the maintainer: an agent in a copy of `examples/tenant` pushes, waits and reads the verdict without parsing text; `shelf app open` serves a private app in the Mac's browser through the devcontainer's forwarded port. +- `/opsx:archive` merges the deltas into `openspec/specs/`; update CLAUDE.md Status, README Status and the context; stop for approval before Phase 10. diff --git a/openspec/config.yaml b/openspec/config.yaml index 17dd972..fbe2dc0 100644 --- a/openspec/config.yaml +++ b/openspec/config.yaml @@ -6,11 +6,20 @@ context: | Keep OpenSpec structural headings and SHALL/MUST keywords in English. ## The product - shelf is a self-hosted app platform for a single Apple Silicon Mac mini, written in Go. Tenants - build images in their own CI and push a deploy artifact to GHCR; shelf watches it and deploys - it. It is an appliance: one binary that is a CLI (`shelf`) and, as `shelf serve`, an admin UI on - the LAN with a login. One-command install and a browser wizard are planned. Learning project for - platform engineering, not for production. + shelf is a self-hosted GitOps platform, written in Go, for running parallel coding-agent + branches as isolated, verified previews on your own hardware, with a path to production through + standard artefacts. Tenants build images in their own CI and push a deploy artifact to GHCR; + shelf watches it and deploys it. One binary is a CLI (`shelf`) and, as `shelf serve`, an admin + UI with a login. Target users: experienced developers and small teams who run coding agents + (Claude Code first) in several worktrees at once, not people who never want to see Kubernetes. + + The target, as re-cut on 2026-10-27: every branch gets its own isolated preview, only a verified + artefact becomes reachable beyond a preview, and the same verified state goes to any Kubernetes + cloud as a pull request (eject), without depending on shelf. shelf runs on any Linux host with + k3s, with a one-step install on a Mac (Colima). Today it runs against any kubecontext, developed + on the dev cluster; previews, verification, the host setup and eject are planned. shelf is + pre-production: an app's "main" is its stable, verified state, not production. Learning and + reference project for platform engineering. OpenSpec is the only planning source: `openspec/specs/` is what shelf does today, `openspec/changes/` what it is about to do (the roadmap is the `phase--` changes), and @@ -20,18 +29,24 @@ context: | proposing, it wins over this summary. ## Architecture - - On the mini (from Phase 9): macOS runs Colima, Colima runs k3s. `shelf serve` runs as the - logged-in user under a LaunchAgent, never as root; its state is plain files under `~/.shelf` - (0700, files 0600): config, admin file, secret backups, Cloudflare connections, audit log. No - keychain: an unsigned, self-updating binary would re-prompt. + - On the host (from Phase 13): a Linux host runs k3s directly; on macOS (Apple Silicon) Colima + runs a Linux VM with k3s, and the same Linux bootstrap follows. Tier 1: macOS with Colima, + Ubuntu/Debian on arm64 and amd64. `shelf serve` runs as the logged-in user, never as root (a + LaunchAgent on macOS); its state is plain files under `~/.shelf` (0700, files 0600): config, + admin file, secret backups, Cloudflare connections, audit log. No keychain: an unsigned, + self-updating binary would re-prompt. - In the cluster: Flux, installed through the Flux Operator (`FluxInstance`), syncs the platform artifact (`platform/`: Traefik as ClusterIP in `traefik`, the ResourceSet `apps` in `shelf-system`). Each app is a `ResourceSetInputProvider`; the ResourceSet renders its namespace, the deploy artifact source and the HelmRelease of the generic chart `shelf-app`. An exposed app has cloudflared in its namespace with one rule to Traefik, which routes by Host. - - `shelf init` is split into idempotent layers: `init host` (the Mac only, Phase 9), - `init cluster` (any kubecontext, no Colima assumptions) and the wrapper. The wizard (Phase 11) - calls the same operations, so re-running it changes nothing. + - `shelf init` is split into idempotent layers: `init host` (Phase 13: provide the environment, + macOS or Linux), `init cluster` (any kubecontext, no Colima or distribution assumptions) and + the wrapper. + - Three separate states per app and preview: build (built, scanned, signed), deployment (which + digest runs where) and exposure (cluster only, port-forward, tunnel). Exposure is the + operator's decision, never part of app.yaml. Previews are reached by port-forward through + the API server to Traefik (`shelf app open`, Phase 9); LAN access stays rejected. - Test levels: 1 is `just test` without a cluster (also CI), 2 the k3d cluster `shelf-dev` in the devcontainer (`just smoke-*`, in CI on pull requests), 3 the mini and real Cloudflare. The rules are in the spec `dev-environment`. @@ -59,18 +74,23 @@ context: | that checks them. ## Not in the MVP - Per-branch environments, scale-to-zero, supply-chain security (signatures, scans, policies), an - own registry, backups, seed data, devcontainer integration for tenant apps, a catalog of managed + Scale-to-zero, an own registry, backups, devcontainer integration for tenant apps, a catalog of managed services; more than one user or roles, admin access from the internet, TLS on the LAN, a stable public HTTP API (`/api/` serves the UI and may change), code signing, notarization, `.pkg` installers, metrics history and graphs, log retention and search, editing app.yaml in the UI, apps without a tenant repository, Prometheus. LAN access to apps (proxy plus mDNS) was - considered and rejected in 8d. + considered and rejected in 8d, and LAN access with wildcard DNS and DNS-01 certificates again + in the re-cut of 2026-10-27; apps and previews are reached by port-forward or a tunnel. ## Later (no phase yet) Packages of a GitHub organisation in the form choices; a short 504 seen once during a rollout of a single-instance app; MongoDB with auth and a replica set (secrets as file mounts); builds via - `path`, CronJobs, init and migration jobs, `${app.url}`, secret rotation. + `path`, CronJobs, init and migration jobs, `${app.url}`, secret rotation. From the re-cut of + 2026-10-27: identity-protected previews through Cloudflare Access; the setup wizard (formerly + Phase 11; re-cut or dropped, its roadmap text is in git history); `--existing-cluster` (install + only shelf's components into an existing cluster, with the eject target preflight); further eject + targets; a go-live check; optional opinionated providers (e.g. `resources: db: {type: + postgres}`); seed data for previews. # Per-artifact rules (optional) # Add custom rules for specific artifacts.