Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/conformance.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ env:
CARGO_TERM_COLOR: always
CONTAINER_ENGINE: docker
KIND_CLUSTER_NAME: praxis-conformance
PRAXIS_IMAGE: ghcr.io/praxis-proxy/praxis:0.5.2
PRAXIS_IMAGE: ghcr.io/praxis-proxy/praxis:0.7.5
OPERATOR_IMAGE: praxis-operator:dev

jobs:
Expand Down
29 changes: 28 additions & 1 deletion .github/workflows/integration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ env:
KIND_CLUSTER_NAME: ci
V: ${{ inputs.debug && '1' || '' }}
OPERATOR_IMAGE: praxis-operator:dev
PRAXIS_IMAGE: ghcr.io/praxis-proxy/praxis:0.5.2
PRAXIS_IMAGE: ghcr.io/praxis-proxy/praxis:0.7.5

jobs:
# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -79,6 +79,33 @@ jobs:
- name: Integration tests
run: make dev-integration

# The monitoring test's disabled phase ran above without the CRD. The
# phases below cover enabled mode with the CRD installed, then disabled
# mode again to prove stale owned resources are removed.
- name: Install PodMonitor CRD
run: |
bash hack/retry.sh curl --fail --location --show-error --silent \
--retry 3 --retry-all-errors --retry-delay 2 \
--output /tmp/monitoring.coreos.com_podmonitors.yaml \
https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v0.80.1/example/prometheus-operator-crd/monitoring.coreos.com_podmonitors.yaml
kubectl apply --server-side -f /tmp/monitoring.coreos.com_podmonitors.yaml

- name: Monitoring enabled phase
env:
PRAXIS_MONITORING_ENABLED: "true"
run: |
kubectl -n praxis-system set env deployment/praxis-operator PRAXIS_MONITORING_ENABLED=true
kubectl -n praxis-system rollout status deployment/praxis-operator --timeout=120s
cargo test --features integration -- --ignored monitoring_reconciliation

- name: Monitoring disabled cleanup phase
env:
PRAXIS_MONITORING_ENABLED: "false"
run: |
kubectl -n praxis-system set env deployment/praxis-operator PRAXIS_MONITORING_ENABLED=false
kubectl -n praxis-system rollout status deployment/praxis-operator --timeout=120s
cargo test --features integration -- --ignored monitoring_reconciliation

- name: Debug cluster state
if: failure()
run: bash hack/dump-debug.sh
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ The format follows [Keep a Changelog], and this project adheres to

### Added

- Opt-in PodMonitor generation and metrics ingress policy with ownership-checked
cleanup when monitoring is disabled or no usable listeners remain.

- Health, readiness and metrics endpoints on port 8080, with matching
probes on the operator Deployment.
- Leader election over a coordination `Lease`, so the operator can run
Expand All @@ -25,6 +28,13 @@ The format follows [Keep a Changelog], and this project adheres to

### Fixed

- Gateway listeners on the reserved health/metrics port 9902 now report
`PortUnavailable` without blocking other valid listeners.
- Generated data-plane configuration now targets Praxis 0.7.5 so health and
metrics bind to the dedicated admin endpoint.
- Gateway listeners without routes now answer HTTP 404 with a static response
instead of generating an empty router chain.

- Status writes no longer restamp unchanged conditions, which had kept
the operator in a permanent reconcile loop.
- Endpoint weight distribution no longer overflows `i32` and aborts the
Expand Down
2 changes: 2 additions & 0 deletions deploy/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ spec:
env:
- name: PRAXIS_IMAGE
value: "__PRAXIS_IMAGE__"
- name: PRAXIS_MONITORING_ENABLED
value: "false"
- name: POD_NAME
valueFrom:
fieldRef:
Expand Down
9 changes: 9 additions & 0 deletions deploy/rbac.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,15 @@ rules:
- apiGroups: ["policy"]
resources: ["poddisruptionbudgets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
# Required even when PRAXIS_MONITORING_ENABLED=false: reconciliation
# checks for stale owned NetworkPolicy and PodMonitor resources. A missing
# Prometheus Operator CRD is treated as already deleted.
- apiGroups: ["networking.k8s.io"]
resources: ["networkpolicies"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["monitoring.coreos.com"]
resources: ["podmonitors"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "patch"]
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ container. Key properties:
- **TLS volumes:** mounts each referenced TLS Secret
as a read-only volume
- **Health probes:** liveness (`/healthy`) and
readiness (`/ready`) on admin port 9901
readiness (`/ready`) on metrics port 9902
- **Security:** runs as UID 100, read-only filesystem,
drops all capabilities except `NET_BIND_SERVICE`,
RuntimeDefault seccomp profile
Expand Down
63 changes: 59 additions & 4 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,51 @@ files for the operator itself.
Container image used for data-plane pods. Set on the
operator Deployment.

- **Default:** `ghcr.io/praxis-proxy/praxis:latest`
- **Deploy manifest:** `ghcr.io/praxis-proxy/praxis:0.3.0`
- **Default:** `ghcr.io/praxis-proxy/praxis:0.7.5`
- **Deploy manifest:** `__PRAXIS_IMAGE__`, substituted with the image chosen
by the deployment tooling (`praxis:dev` for local development).

```yaml
env:
- name: PRAXIS_IMAGE
value: "ghcr.io/praxis-proxy/praxis:0.3.0"
value: "ghcr.io/praxis-proxy/praxis:0.7.5"
```

The data plane reserves port 9902 for health and metrics. A Gateway listener
requesting that port reports `Accepted=False` with reason `PortUnavailable`;
other valid listeners remain supported. If no usable listeners remain, including
when all listeners conflict, the operator removes the Gateway’s managed proxy
resources and recreates them when a usable listener is restored.

### Optional monitoring resources

Set `PRAXIS_MONITORING_ENABLED=true` to have the operator create a
`PodMonitor` and a metrics-only `NetworkPolicy` for each managed data plane.
The Prometheus Operator CRD must be installed before enabling this option;
with the option unset or false, reconciliation tolerates an absent CRD and
removes any owned monitoring resources left by an earlier enabled configuration.

- **Default:** disabled
- **Namespace:** `PRAXIS_MONITORING_NAMESPACE` (default `monitoring`)
- **Prometheus selector:** `PRAXIS_MONITORING_POD_SELECTOR` as comma-separated
`key=value` labels (default `app.kubernetes.io/name=prometheus`). Malformed or
duplicate requirements cause a configuration error; no partial selector is applied.

The generated policy allows only the selected Prometheus pods in the selected
namespace to reach port 9902. It explicitly allows TCP traffic to every configured
data listener port from any source; Kubernetes combines these allowances with
other policies additively. If the
cluster's CNI applies NetworkPolicy to node-originated kubelet probes, add the
cluster's node CIDRs in a separate policy overlay.

```yaml
env:
- name: PRAXIS_MONITORING_ENABLED
value: "true"
- name: PRAXIS_MONITORING_NAMESPACE
value: observability
- name: PRAXIS_MONITORING_POD_SELECTOR
value: "app.kubernetes.io/name=prometheus,app.kubernetes.io/instance=main"
```

### `RUST_LOG`
Expand Down Expand Up @@ -61,6 +99,23 @@ permissions:
| secrets | get, list, watch, create, update, patch, delete |
| events | create, patch |

### Networking Resources

| API group | Resource | Verbs |
|-----------|----------|-------|
| networking.k8s.io | networkpolicies | get, list, watch, create, update, patch, delete |

### Monitoring Operator Resources

| API group | Resource | Verbs |
|-----------|----------|-------|
| monitoring.coreos.com | podmonitors | get, list, watch, create, update, patch, delete |

> **Upgrading:** apply the updated `deploy/rbac.yaml` before updating the
> operator image, even when monitoring is disabled. Disabled-mode
> reconciliation still checks for and removes stale owned `NetworkPolicy` and
> `PodMonitor` resources, and the previous ClusterRole lacks those permissions.

### Discovery Resources

| Resource | Verbs |
Expand Down Expand Up @@ -153,7 +208,7 @@ directly.

- **ConfigMap:** `praxis-{gateway-name}` with key
`config.yaml`
- **Admin port:** 9901 (health probes, diagnostics)
- **Admin port:** 9902 (health and metrics; `admin.metrics_address`)
- **Config changes:** a SHA-256 hash annotation
(`praxis.sh/config-hash`) on the pod template
triggers a rolling update when config changes
Expand Down
2 changes: 1 addition & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ make kind-down
| Variable | Default |
|---------------------|---------------------------------|
| `KIND_CLUSTER_NAME` | `praxis-conformance` |
| `PRAXIS_IMAGE` | `ghcr.io/praxis-proxy/praxis:0.3.0` |
| `PRAXIS_IMAGE` | `praxis:dev` |
| `OPERATOR_IMAGE` | `praxis-operator:dev` |

## Testing
Expand Down
Loading
Loading