[BUG] compose watch does not work with Enhanced Container Isolation on Docker Desktop MacOS Enterprise

[adonh]

Description

compose watch does not work with Enhanced Container Isolation on Docker Desktop MacOS Enterprise.

Steps To Reproduce

with Docker Desktop MacOS Enterprise:
Follow the same repository that demonstrates compose watch: https://github.com/dockersamples/avatars
Try with and without ECI.
Compose watch does not work with ECI enabled.

Compose Version

docker compose: Docker Compose version v5.1.3
docker-compose: Docker Compose version v5.1.3

Docker Environment

Client:
 Version:    29.4.3
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  agent: Docker AI Agent Runner (Docker Inc.)
    Version:  v1.57.0
    Path:     /Users/.../.docker/cli-plugins/docker-agent
  ai: Docker AI Agent - Ask Gordon (Docker Inc.)
    Version:  v1.20.2
    Path:     /Users/.../.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.33.0-desktop.1
    Path:     /Users/.../.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v5.1.3
    Path:     /Users/.../.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.47
    Path:     /Users/.../.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Docker Inc.)
    Version:  v0.3.0
    Path:     /Users/.../.docker/cli-plugins/docker-desktop
  dhi: CLI for managing Docker Hardened Images (Docker Inc.)
    Version:  v0.0.3
    Path:     /Users/.../.docker/cli-plugins/docker-dhi
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.31
    Path:     /Users/.../.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /Users/.../.docker/cli-plugins/docker-init
  mcp: Docker MCP Plugin (Docker Inc.)
    Version:  v0.42.1
    Path:     /Users/.../.docker/cli-plugins/docker-mcp
  model: Docker Model Runner (Docker Inc.)
    Version:  v1.1.37
    Path:     /Users/.../.docker/cli-plugins/docker-model
  offload: Docker Offload (Docker Inc.)
    Version:  v0.5.89
    Path:     /Users/.../.docker/cli-plugins/docker-offload
  pass: Docker Pass Secrets Manager Plugin (beta) (Docker Inc.)
    Version:  v0.0.27
    Path:     /Users/.../.docker/cli-plugins/docker-pass
  sandbox: Docker Sandbox (Docker Inc.)
    Version:  v0.12.0
    Path:     /Users/.../.docker/cli-plugins/docker-sandbox
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/.../.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.20.4
    Path:     /Users/.../.docker/cli-plugins/docker-scout

Server:
 Containers: 5
  Running: 1
  Paused: 0
  Stopped: 4
 Images: 28
 Server Version: 29.4.3
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Discovered Devices:
  cdi: docker.com/gpu=webgpu
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 77c84241c7cbdd9b4eca2591793e3d4f4317c590
 runc version: v1.3.5-0-g488fc13e
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.12.76-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 8
 Total Memory: 7.75GiB
 Name: docker-desktop
 ID: 1b6fff06-aa47-4f54-bd13-f908199a17cb
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/.../Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false
 Firewall Backend: iptables

Anything else?

Error upon doing touch /app/api/ha outside the container on the host:

[+] up 3/3
 ✔ Image avatars-api       Built                                                                                                                                                0.8s
 ✔ Network avatars_default Created                                                                                                                                              0.0s
 ✔ Container avatars-api-1 Created                                                                                                                                              0.0s
Watch enabled
Syncing service "api" after 1 changes were detected
WARN[0033] Error handling changed files: copying files to 0c6097adac319863fcde031c89ff85f7073876d9bf08f89fe09cec47bcac1c4a: Error response from daemon: open /app/api/ha: directory not empty

[ndeloof]

This is not a compose specific issue: watch relies on Moby's CopyToContainer API, and with ECI this API is broken:

$ docker cp api/app.py avatars-api-1:/app/api/app.py
Successfully copied 2.91kB (transferred 4.61kB) to avatars-api-1:/app/api/app.py
Error response from daemon: unlinkat /app/api/app.py: directory not empty

This can be reproduced with Docker CE + Sysbox
Created nestybox/sysbox#1017

cc @ctalledo

[ctalledo]

Hi @adonh, what version of Docker Desktop are you running please?

FYI, there was a Docker Desktop regression in "docker cp" with ECI in 4.70; it has been fixed internally, the fix will be available in the upcoming 4.77 release.

[ctalledo]

@ndeloof: Thanks for reproducing with Docker CE + Sysbox and filing nestybox/sysbox#1017.

Note however that Docker Desktop behaves a bit different from Docker CE, and "docker cp" with ECI works in Docker Desktop, except there was a regression in DD 4.70 (see my prior comment).