Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/typescript-node
{
"name": "Node.js & TypeScript",
// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
"image": "mcr.microsoft.com/devcontainers/typescript-node:26-trixie",
"features": {
"ghcr.io/devcontainers/features/github-cli:1": {
"installDirectlyFromGitHubRelease": true,
"version": "latest"
},
"ghcr.io/devcontainers/features/kubectl-helm-minikube:1": {
"version": "latest",
"helm": "latest",
"minikube": "none"
}
},
"customizations": {
"vscode": {
"extensions": [
"anthropic.claude-code"
]
}
}

// Features to add to the dev container. More info: https://containers.dev/features.
// "features": {},

// Use 'forwardPorts' to make a list of ports inside the container available locally.
// "forwardPorts": [],

// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "yarn install",

// Configure tool-specific properties.
// "customizations": {},

// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
// "remoteUser": "root"
}
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,5 @@ body:
placeholder: |
helm version: v3.x
kubectl version: v1.x
node version: v20.x
node version: v22.x
cluster: kind / EKS / GKE / ...
2 changes: 1 addition & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
node-version-file: .nvmrc
- run: npm install
- run: npx tsc --noEmit
- run: npx cucumber-js --dry-run
Expand Down
33 changes: 26 additions & 7 deletions .github/workflows/real-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,16 @@ on:
branches: [main]
pull_request:

# Two levels:
# - per ref (this block): a newer push to the same PR cancels that PR's
# older run, running or still queued, so a stale commit never holds
# the one shared slot below. Pushes to main are never cancelled.
# - global (the test job's own `concurrency`): thomas-helm-test is one
# shared, fixed namespace - never run two suites against it at once.
# A cancel is graceful, so the `if: always()` uninstall step still runs.
concurrency:
group: real-tests
cancel-in-progress: false # thomas-helm-test is one shared, fixed namespace — never run two suites against it at once
group: real-tests-${{ github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

jobs:
test:
Expand All @@ -26,12 +33,24 @@ jobs:
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name == github.repository
runs-on: thomas
timeout-minutes: 30
concurrency:
group: real-tests
cancel-in-progress: false
steps:
- uses: actions/checkout@v4

# Node 26's official Linux build links libatomic.so.1, which the
# self-hosted runner image doesn't ship (GitHub-hosted runners do).
- name: Install libatomic1 for Node 26
run: |
if ! ldconfig -p | grep -q 'libatomic\.so\.1'; then
sudo apt-get update -qq
sudo apt-get install -y -qq libatomic1
fi

- uses: actions/setup-node@v4
with:
node-version: "20"
node-version-file: .nvmrc

# scripts/tools.sh is the single source of truth for tool
# versions - the same script a contributor runs locally via `npm
Expand All @@ -44,11 +63,11 @@ jobs:
echo "$HOME/.local/bin" >> "$GITHUB_PATH"

- run: npm install
# Excludes @requires-broad-rbac (currently just quickstart.feature)
# Excludes @requires-broad-rbac (quickstart.feature, features/docker/)
# - the CI ServiceAccount is intentionally scoped to
# thomas-helm-test only (see docs/project/ci.md), and that
# scenario deliberately deploys into a different namespace on
# purpose. Still runs locally for anyone with broader access.
# thomas-helm-test only (see docs/project/ci.md), and those
# scenarios need access outside it. Still run locally for anyone
# with broader access.
- run: npm run test:verbose -- --tags "not @requires-broad-rbac"

# TODO: I think namespace should be unique per run, not a fixed thomas-helm-test,
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-prepare.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:

- uses: actions/setup-node@v4
with:
node-version: "20"
node-version-file: .nvmrc

- name: Compute release version
id: version
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:

- uses: actions/setup-node@v4
with:
node-version: "20"
node-version-file: .nvmrc

- name: Determine expected version from the branch name
id: expected
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/version-bump.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:

- uses: actions/setup-node@v4
with:
node-version: "20"
node-version-file: .nvmrc

- name: Bump patch version if needed
id: bump
Expand Down
1 change: 1 addition & 0 deletions .nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
26
10 changes: 6 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -246,10 +246,12 @@ with MkDocs + Material).
real image, scoped `--registry-credentials` via a `runCommand` `env`
option) — shipped, verified. Resources: 20/20 scenarios passing.
Docker: 12/12 scenarios passing against a real, disposable BuildKit +
registry pair in `thomas-helm-test` (which required labeling that
namespace `pod-security.kubernetes.io/enforce: privileged` — BuildKit
needs privileged/rootless-unconfined execution the default `baseline`
level blocks), run twice in a row, cluster confirmed clean both times
registry pair in `thomas-helm-test` (the scenarios themselves label
that namespace `pod-security.kubernetes.io/enforce=privileged` before
installing BuildKit and remove it after — BuildKit needs
privileged/rootless-unconfined execution the default `baseline` level
blocks; tagged `@requires-broad-rbac`, so skipped in CI — see
`docs/project/ci.md`), run twice in a row, cluster confirmed clean both times
(allowing a few seconds for normal terminating-Pod GC lag after
`helm uninstall --wait` — a transient "Error" status during container
shutdown, not a real leftover, see `docker-buildx-builder-full.feature`'s
Expand Down
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/).

## [Unreleased]

### Changed

- **`@cucumber/cucumber` 13** (peer `^13.2.1`, was `^12.2.0`). Its `--parallel`
workers are worker threads now, which don't get the `tsx` loader that
`cucumber.mjs` registers: every `test*` script sets
`NODE_OPTIONS='--import tsx'` (consumers running in parallel need the same).
`engines` is `node >=26` (cucumber 13 dropped Node 20); CI, the dev
container and `@types/node` all track Node 26, via `.nvmrc` in CI.

### Added

- **Every `assertCondition` failure now points at the current scenario's
Expand Down
18 changes: 16 additions & 2 deletions docs/project/ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,13 @@ open one, and the runner pod's own in-cluster identity has real write
access to the cluster (see below) — not something a fork PR's workflow
run is trusted with.

`concurrency: {group: real-tests, cancel-in-progress: false}` —
serializes every run. `thomas-helm-test` is one fixed, shared
The `test` job's `concurrency: {group: real-tests, cancel-in-progress:
false}` serializes every run. A second, workflow-level group per ref
(`real-tests-${{ github.ref }}`, `cancel-in-progress` on pull requests
only) cancels a PR's older run — running or still queued — when a newer
commit is pushed, so a stale commit never holds that one slot; the
graceful cancel still runs the `if: always()` uninstall step. Pushes to
`main` are never cancelled. `thomas-helm-test` is one fixed, shared
namespace (not one per run), and this cluster has already shown real
fragility under concurrent/shared use elsewhere on it — see
`rts-turbo`'s own `ci.yml` for the same tradeoff made the same way.
Expand Down Expand Up @@ -125,6 +130,15 @@ namespace just for one demo scenario isn't worth trading away the
narrow-RBAC guarantee above for — this scenario still runs for anyone
with broader cluster access, same as before.

Both `features/docker/` features carry it too: they label
`thomas-helm-test` itself `pod-security.kubernetes.io/enforce=privileged`
for BuildKit's privileged pod (see [DOCKER.md](../reference/DOCKER.md)).
A `Namespace` is a cluster-scoped object, which the namespaced `Role`
above can't grant anything on — running them in CI would need an extra
`ClusterRole`/`ClusterRoleBinding` (`get`/`patch` on `namespaces`,
`resourceNames: ["thomas-helm-test"]`, bound to
`arc-runners:thomas-ci`), deliberately not set up for now.

## Tool versions: `scripts/tools.sh` (`npm run install:tools`)

Both workflows install `helm`/`kubectl` via `npm run install:tools`
Expand Down
16 changes: 11 additions & 5 deletions docs/reference/DOCKER.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,9 +146,15 @@ via `fixtures/docker/buildkitd.toml` to trust that registry as plain
HTTP) into `thomas-helm-test`, then build and push a real image
(`fixtures/docker/Dockerfile`) end to end.

**Prerequisite**: a real BuildKit pod needs privileged (or
**PodSecurity**: a real BuildKit pod needs privileged (or
rootless-but-unconfined) execution, which the cluster's default
`baseline` PodSecurity level blocks. `thomas-helm-test` is labeled
`pod-security.kubernetes.io/enforce: privileged` to allow this — a
one-time, namespace-scoped setup step, not something these feature
files do themselves.
`baseline` PodSecurity level blocks. Each scenario labels
`thomas-helm-test` `pod-security.kubernetes.io/enforce=privileged`
([`I label namespace`](KUBECTL.md)) right before installing BuildKit,
and removes the label again right after uninstalling it — so a
recreated namespace needs no manual setup, and it only stays privileged
for as long as BuildKit runs (unless a scenario fails mid-way, in which
case the label is left behind until the next successful run). Labeling
a namespace needs cluster-scoped access CI's `ServiceAccount` doesn't
have, so both features are tagged `@requires-broad-rbac` and skipped in
CI — see [CI](../project/ci.md#requires-broad-rbac-scenarios-that-deliberately-cant-run-here).
7 changes: 4 additions & 3 deletions docs/reference/HELM.md
Original file line number Diff line number Diff line change
Expand Up @@ -587,8 +587,9 @@ else.

!!! note "`helm install` is genuinely not idempotent"
A second install of the same release name errors
(`cannot re-use a name that is still in use`) — that's real, correct
Helm behavior, not a framework limitation. The rerun-safe idiom used
(`cannot reuse a name that is still in use` — Helm v3 spelled it
`re-use`, so the scenarios match only the shared tail) — that's real,
correct Helm behavior, not a framework limitation. The rerun-safe idiom used
throughout this suite is `upgrade` with `--install`/`--atomic` set
(see the "upgrade" tab below).

Expand Down Expand Up @@ -634,7 +635,7 @@ registered Release first.
When I install Helm Release "<DuplicateRelease>" with --create-namespace
Then the command exited with 0
When I install Helm Release known as "<DuplicateRelease>"
Then the command exited with 1 STDERR contains "cannot re-use a name"
Then the command exited with 1 STDERR contains "name that is still in use"
```

=== "Full"
Expand Down
16 changes: 15 additions & 1 deletion features/docker/docker-buildx-builder-full.feature
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Labels thomas-helm-test itself (a cluster-scoped Namespace object) for
# BuildKit's privileged pod, which the narrowly-scoped CI ServiceAccount
# can't do - see docs/project/ci.md.
@requires-broad-rbac
Feature: BDD Framework for Docker Buildx Builders (full table syntax)
As a platform engineer
I want to create, use, and remove a real `docker buildx` builder backed
Expand Down Expand Up @@ -33,7 +37,13 @@ Feature: BDD Framework for Docker Buildx Builders (full table syntax)
# test registry as plain HTTP - no mocks, and no dependency on any
# BuildKit instance outside Thomas's own control. Needs a namespace
# whose PodSecurity level allows a privileged (or rootless-unconfined)
# pod - see docs/reference/DOCKER.md.
# pod, so the scenario raises it here and drops it back after
# uninstalling BuildKit - see docs/reference/DOCKER.md.
When I label namespace "thomas-helm-test" with:
| OPTION | VALUE |
| | pod-security.kubernetes.io/enforce=privileged |
| --overwrite | True |
Then the command exited with 0
Given Helm Chart known as "<BuildkitChart>":
| PROPERTY | VALUE |
| chart | buildkit-service |
Expand Down Expand Up @@ -67,6 +77,10 @@ Feature: BDD Framework for Docker Buildx Builders (full table syntax)
| OPTION | VALUE |
| --wait | True |
Then the command exited with 0
When I label namespace "thomas-helm-test" with:
| OPTION | VALUE |
| | pod-security.kubernetes.io/enforce- |
Then the command exited with 0
When I uninstall Helm Release known as "<TestRegistryRelease>" with:
| OPTION | VALUE |
| --wait | True |
Expand Down
10 changes: 9 additions & 1 deletion features/docker/docker-buildx-builder-short.feature
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Labels thomas-helm-test itself (a cluster-scoped Namespace object) for
# BuildKit's privileged pod, which the narrowly-scoped CI ServiceAccount
# can't do - see docs/project/ci.md.
@requires-broad-rbac
Feature: BDD Framework for Docker Buildx Builders (short syntax)
As a platform engineer
I want to create, use, and remove a real `docker buildx` builder backed
Expand All @@ -17,7 +21,9 @@ Feature: BDD Framework for Docker Buildx Builders (short syntax)
# A real, disposable BuildKit instance trusting our real, disposable
# test registry as plain HTTP - see docker-buildx-builder-full.feature's
# Background for why, and docs/reference/DOCKER.md for the namespace
# PodSecurity prerequisite.
# PodSecurity label raised here and dropped after uninstalling BuildKit.
When I label namespace "thomas-helm-test" with --overwrite pod-security.kubernetes.io/enforce=privileged
Then the command exited with 0
Given Helm Chart known as "<BuildkitChart>":
| PROPERTY | VALUE |
| chart | buildkit-service |
Expand All @@ -36,5 +42,7 @@ Feature: BDD Framework for Docker Buildx Builders (short syntax)
Then the command exited with 0
When I uninstall Helm Release "<BuildkitRelease>" with --wait
Then the command exited with 0
When I label namespace "thomas-helm-test" with pod-security.kubernetes.io/enforce-
Then the command exited with 0
When I uninstall Helm Release "<TestRegistryRelease>" with --wait
Then the command exited with 0
4 changes: 2 additions & 2 deletions features/helm/release-full.feature
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ Feature: BDD Framework for Helm Releases (full table syntax)
Then the command exited with 0
When I install Helm Release known as "<DuplicateRelease>"
Then the command exited with 1:
| SOURCE | CONDITION | VALUE |
| STDERR | contains | cannot re-use a name |
| SOURCE | CONDITION | VALUE |
| STDERR | contains | name that is still in use |
When I uninstall Helm Release known as "<DuplicateRelease>"
Then the command exited with 0
2 changes: 1 addition & 1 deletion features/helm/release-short.feature
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,6 @@ Feature: BDD Framework for Helm Releases (short syntax)
When I install Helm Release "<DuplicateRelease>" with --create-namespace
Then the command exited with 0
When I install Helm Release known as "<DuplicateRelease>"
Then the command exited with 1 STDERR contains "cannot re-use a name"
Then the command exited with 1 STDERR contains "name that is still in use"
When I uninstall Helm Release known as "<DuplicateRelease>"
Then the command exited with 0
Loading
Loading