eUPF can run as a non-root user with file capabilities; today the chart runs it as root #671

Today

The chart runs the eupf container as root with the documented capability set (NET_ADMIN, SYS_ADMIN, SYS_RESOURCE, BPF, PERFMON, since #666). Root isn't needed for what eUPF does: loading the programs needs BPF and PERFMON on the kernels the chart supports, attaching XDP needs NET_ADMIN, raising RLIMIT_MEMLOCK needs SYS_RESOURCE. The one thing uid 0 does that the capabilities don't is creating the map pin directory, /sys/fs/bpf/upf_pipeline (cmd/ebpf/objects.go): bpffs's root directory belongs to root, mode 0700.

A root container is what every Kubernetes hardening guide asks to avoid (Pod Security Standards restricted, CIS 5.2.6, NSA/CISA), and non_root_containers is one of the three failures the chart has left in the CNTi essential tests.

What works

Tested on a fork (run 38950181467: the 0.7.1 source, kind cluster, the CNTi suite's 19 essential tests):

1. File capabilities on the binary, in the runtime stage of the Dockerfile:

   RUN apk add --no-cache libcap \
       && setcap cap_net_admin,cap_sys_resource,cap_bpf,cap_perfmon,cap_dac_override=ep /app/bin/eupf \
       && apk del libcap

   DAC_OVERRIDE is what lets uid 1000 create the pin directory. SYS_ADMIN isn't in the set: a non-root entrypoint mounts nothing, and the chart requires kernel 5.15, which has CAP_BPF. One rule to document: a binary with file capabilities fails to exec (Operation not permitted) when the container's bounding set lacks one of them, as root or not; the chart's defaults grant them, docker run needs --privileged or --cap-add.
2. The chart: runAsUser: 1000, runAsGroup: 1000, runAsNonRoot: true, capabilities.add: [NET_ADMIN, SYS_RESOURCE, BPF, PERFMON, DAC_OVERRIDE], and only /sys/fs/bpf mounted from the node instead of all of /sys. allowPrivilegeEscalation stays unset (file capabilities need no_new_privs off).
3. bpffs mounted on the node. As root, entrypoint.sh mounts bpffs itself when it's missing; a non-root process can't, so the node has to provide it, as every systemd host does (sys-fs-bpf.mount) and as other eBPF workloads require. kind nodes don't; for the CI cluster it's bound in from the runner with an extraMounts entry.

Result: eUPF runs as uid 1000 with exactly those five effective capabilities (CapEff 000000c001001002), less than root with the current set, attaches XDP, and the suite's non_root_containers test passes: 15 → 16 of 19.

The one blunt part

DAC_OVERRIDE stands in for a single mkdir in a root-owned directory, but it bypasses permission checks on every file the process can reach. Only the code can make that cleaner: a configurable pin directory, or pinning that can be turned off (the maps then live with the process). That's your call; DAC_OVERRIDE is the chart-side way that needs no code change.

Sequence

The chart's default can only change once an image carries the file capabilities; runAsUser: 1000 with today's 0.7.1 image would fail to start. So: the image first (Dockerfile, the entrypoint mounting only as root; no change for current users), then, with the next image release, the chart's appVersion bump together with the non-root default and the bpffs note in the README and values.yaml.

Related: #670 (main and the chart's memory limit), found in the same runs.