Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .icons/openshift.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
89 changes: 89 additions & 0 deletions registry/coder/templates/openshift/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
display_name: OpenShift
description: Provision OpenShift workspaces with arbitrary UIDs and persistent home directories under restricted-v2
icon: ../../../../.icons/openshift.svg
verified: false
tags: [openshift, kubernetes, container]
---

# OpenShift

Provision an OpenShift Deployment as a Coder workspace with a persistent home directory and code-server. Based on the [Kubernetes template](../kubernetes), this template lets OpenShift assign the workspace's user ID and volume group under the default `restricted-v2` Security Context Constraint (SCC).

## Prerequisites

Use an existing OpenShift project with `restricted-v2` available to its `default` service account, enough resource quota for the workspace, and a default StorageClass that provisions `ReadWriteOnce` volumes. The storage driver must honor the SCC-assigned `fsGroup` so a newly mounted home volume is writable. Set a different StorageClass when the cluster has no suitable default. NFS exports with fixed ownership may require administrator configuration; do not work around storage permissions by granting a privileged SCC.

Install the OpenShift CLI (`oc`) and the Coder CLI on your administration machine. Authenticate to the cluster and select the workspace project:

```sh
oc login <cluster-api-url>
oc project <workspace-project>
oc auth can-i use scc/restricted-v2 --as=system:serviceaccount:<workspace-project>:default
```

Checking another service account may require impersonation permission; ask your cluster administrator to verify SCC access if needed. No SCC changes are required by this template.

The **Coder provisioner**, rather than your local CLI, needs cluster credentials. When it runs inside the cluster, use its service account. When it runs outside, enable kubeconfig authentication and provide `~/.kube/config` on the provisioner's host. That identity needs `get`, `list`, `watch`, `create`, `update`, `patch`, and `delete` on `deployments.apps` and `persistentvolumeclaims` in the target project. Read access to pods, events, and pod logs is useful for troubleshooting. The workspace service account does not need those provisioning privileges, and its API token is not mounted in workspace pods.

The provisioner must reach the OpenShift API. Workspace pods must reach your Coder access URL for agent download and connection. Nodes need access to `quay.io` for the default image; code-server installation needs `code-server.dev`, GitHub, and GitHub release downloads. Adapt images and module installation to your organization's network policy before using a restricted network.

## Workspace image

The default [Devfile base developer image](https://github.com/devfile/developer-images/tree/main/base/ubi9) uses Red Hat UBI 9 and is pinned to an image digest. It provides Bash, curl, tar, Git, and an entrypoint that registers the arbitrary UID supplied by OpenShift. Its home directory is `/home/user`, writable by group 0. The template supplies agent bootstrap through container `args`, preserving that entrypoint; replacing `command` would bypass UID initialization.

You can substitute an image that supports the same home path, arbitrary UID, and entrypoint contract. Select a matching workspace architecture; the template schedules the pod onto Linux nodes with that architecture. The default is AMD64, with ARM64 also supported. Tools must be installed into the image or under the home directory: package installation with sudo and privileged container engines are not supported under `restricted-v2`.

See [Red Hat's image authoring guidance](https://docs.redhat.com/en/documentation/openshift_container_platform/4.19/html/images/creating-images#use-uid_create-images) for the root-group permissions pattern. Image directory permissions alone do not set permissions on a mounted PVC; the storage requirement above still applies.

## Architecture

The Deployment and code-server app exist only while the workspace is started. The Deployment uses `Recreate` to avoid simultaneous pods attaching the same home volume. CPU and memory limits can be changed when rebuilding a workspace.

A separate PVC is mounted at `/home/user` and remains when the workspace is stopped. Its configuration is ignored after creation to avoid replacement when workspace metadata or template defaults change. Files outside the home directory are ephemeral. Deleting the workspace deletes its PVC; the StorageClass reclaim policy determines whether the underlying storage is retained. Back up important files before deletion.

The template does not specify `run_as_user`, `run_as_group`, or `fs_group`. OpenShift assigns these at admission. The pod requests non-root execution and `RuntimeDefault` seccomp, drops all container capabilities, and disables privilege escalation. It does not create SCCs or grant `anyuid`. See [OpenShift SCC documentation](https://docs.redhat.com/en/documentation/openshift_container_platform/4.19/html/authentication_and_authorization/managing-pod-security-policies) and [Coder on OpenShift](https://coder.com/docs/install/server/openshift).

## Validation

Tested on OpenShift **4.21.30** (Red Hat Developer Sandbox on ROSA, AMD64) with Coder **2.36.7**, Terraform **1.16.4**, and the default image digest above. The cluster's default `gp3` StorageClass provisioned a 10 GiB home volume. The pod was admitted under `restricted-v2` with an OpenShift-assigned UID and volume group; the agent connected, and both the Coder terminal and code-server worked.

Stopping removed the Deployment and pod while preserving the same PVC. After restarting, the agent reconnected and a file created in the terminal and edited in code-server retained its contents. Deleting the disposable workspace removed its Deployment, pod, and PVC. Other OpenShift versions, storage drivers, and ARM64 clusters have not been tested end to end.

Run local configuration checks from this directory:

```sh
terraform init
terraform validate
terraform test
```

The Terraform tests use mocked providers to check security settings, entrypoint preservation, start/stop resource selection, and input validation. They do not connect to a cluster.

To repeat validation on your OpenShift cluster:

1. Record `oc version` (including the server version), Coder version, image digest, and StorageClass. Push the template to a test Coder deployment from the repository root:

```sh
coder templates push openshift -d registry/coder/templates/openshift -m "OpenShift workspace validation"
```

Supply the existing project and the provisioner's authentication mode when prompted. Create a workspace using the template and wait for the agent to connect.

2. Find its pod by workspace ID and verify the admitted SCC and assigned UID. Use the workspace's `id` from `coder list --output json`:

```sh
WORKSPACE_ID=<workspace-uuid>
POD=$(oc get pods -l "com.coder.workspace.id=$WORKSPACE_ID" -o jsonpath='{.items[0].metadata.name}')
oc get pod "$POD" -o jsonpath='{.metadata.annotations.openshift\.io/scc}{"\n"}'
oc get pod "$POD" -o jsonpath='{.spec.containers[0].securityContext.runAsUser}{"\n"}'
oc get namespace "$(oc project -q)" -o jsonpath='{.metadata.annotations.openshift\.io/sa\.scc\.uid-range}{"\n"}'
```

Confirm `restricted-v2` and a UID in the project's allocated range. A more privileged SCC is not a valid result.

3. Open the Coder web terminal and code-server. In the terminal, run `id`, `getent passwd "$(id -u)"`, and `printf 'persistent\n' > "$HOME/openshift-persistence-check"`. Confirm the editor can open and modify a file in the home directory.

4. Stop the workspace. Confirm its Deployment and pods are gone but its PVC remains. Restart it, confirm the agent reconnects, and verify the marker and edited file contents. Delete only this disposable test workspace and confirm its Deployment, pods, and PVC are gone.

5. Record the tested OpenShift version and observed results, including any differences from the environment documented above.
219 changes: 219 additions & 0 deletions registry/coder/templates/openshift/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
terraform {
required_providers {
coder = {
source = "coder/coder"
}
kubernetes = {
source = "hashicorp/kubernetes"
}
}
}

provider "coder" {}

provider "kubernetes" {
config_path = var.use_kubeconfig ? "~/.kube/config" : null
}

data "coder_workspace" "me" {}
data "coder_workspace_owner" "me" {}

locals {
home_directory = "/home/user"
workspace_labels = {
"app.kubernetes.io/name" = "coder-workspace"
"app.kubernetes.io/instance" = "coder-workspace-${data.coder_workspace.me.id}"
"app.kubernetes.io/part-of" = "coder"
"com.coder.resource" = "true"
"com.coder.workspace.id" = data.coder_workspace.me.id
"com.coder.workspace.name" = data.coder_workspace.me.name
"com.coder.user.id" = data.coder_workspace_owner.me.id
"com.coder.user.username" = data.coder_workspace_owner.me.name
}
}

resource "coder_agent" "main" {
os = "linux"
arch = var.arch

metadata {
display_name = "CPU Usage"
key = "0_cpu_usage"
script = "coder stat cpu"
interval = 10
timeout = 1
}

metadata {
display_name = "RAM Usage"
key = "1_ram_usage"
script = "coder stat mem"
interval = 10
timeout = 1
}

metadata {
display_name = "Home Disk"
key = "2_home_disk"
script = "coder stat disk --path ${local.home_directory}"
interval = 60
timeout = 1
}
}

module "code-server" {
count = data.coder_workspace.me.start_count
source = "registry.coder.com/coder/code-server/coder"
version = "1.6.0"
agent_id = coder_agent.main.id
order = 1
folder = local.home_directory
}

resource "kubernetes_persistent_volume_claim_v1" "home" {
metadata {
name = "coder-${data.coder_workspace.me.id}-home"
namespace = var.namespace
labels = local.workspace_labels
annotations = {
"com.coder.user.email" = data.coder_workspace_owner.me.email
}
}
wait_until_bound = false
spec {
access_modes = ["ReadWriteOnce"]
storage_class_name = var.storage_class_name == "" ? null : var.storage_class_name
resources {
requests = {
storage = "${data.coder_parameter.home_disk_size.value}Gi"
}
}
}

lifecycle {
ignore_changes = all
}
}

resource "kubernetes_deployment_v1" "main" {
count = data.coder_workspace.me.start_count
wait_for_rollout = false
metadata {
name = "coder-${data.coder_workspace.me.id}"
namespace = var.namespace
labels = local.workspace_labels
annotations = {
"com.coder.user.email" = data.coder_workspace_owner.me.email
}
}

spec {
replicas = 1
selector {
match_labels = {
"com.coder.workspace.id" = data.coder_workspace.me.id
}
}
strategy {
type = "Recreate"
}

template {
metadata {
labels = local.workspace_labels
}
spec {
service_account_name = "default"
automount_service_account_token = false
node_selector = {
"kubernetes.io/arch" = var.arch
"kubernetes.io/os" = "linux"
}
security_context {
run_as_non_root = true
seccomp_profile {
type = "RuntimeDefault"
}
}

container {
name = "dev"
image = var.image
image_pull_policy = "IfNotPresent"
# Preserve the image entrypoint: it registers the OpenShift-assigned UID.
args = ["sh", "-c", coder_agent.main.init_script]
working_dir = local.home_directory
security_context {
allow_privilege_escalation = false
run_as_non_root = true
capabilities {
drop = ["ALL"]
}
}
env {
name = "CODER_AGENT_TOKEN"
value = coder_agent.main.token
}
env {
name = "HOME"
value = local.home_directory
}
resources {
requests = {
cpu = "250m"
memory = "512Mi"
}
limits = {
cpu = data.coder_parameter.cpu.value
memory = "${data.coder_parameter.memory.value}Gi"
}
}
volume_mount {
mount_path = local.home_directory
name = "home"
read_only = false
}
}

volume {
name = "home"
persistent_volume_claim {
claim_name = kubernetes_persistent_volume_claim_v1.home.metadata[0].name
read_only = false
}
}

affinity {
pod_anti_affinity {
preferred_during_scheduling_ignored_during_execution {
weight = 1
pod_affinity_term {
topology_key = "kubernetes.io/hostname"
label_selector {
match_expressions {
key = "app.kubernetes.io/name"
operator = "In"
values = ["coder-workspace"]
}
}
}
}
}
}
}
}
}
}

resource "coder_metadata" "workspace" {
count = data.coder_workspace.me.start_count
resource_id = kubernetes_deployment_v1.main[count.index].id
item {
key = "image"
value = var.image
}
item {
key = "project"
value = var.namespace
}
}
Loading
Loading