Windows: missing secrets file creates file-named directory [again]

[filimonic]

docker compose up creates folder named as secret file when file does not exist

> docker compose version
Docker Compose version v2.40.0-desktop.1
> docker compose up

name: test
secrets:
  sample_secret:
    file: ./sample.secret
services:
  sample:
    image: "alpine:latest"
    secrets:
      - sample_secret

I'm still seeing this problem with Docker Compose version v2.32.4-desktop.1 on Windows.

Originally posted by @Kylotan in #8305

[ndeloof]

Don't you get a warning ?

$ docker compose up
WARN[0000] secret file xx_sample_secret does not exist 

[Grimeh]

Also seeing this on Windows.

For me the issue isn't whether there's a warning or not but the fact that it creates an unwanted folder which must be manually deleted. This doesn't happen on other platforms afaik.
Ideally it would error out immediately if a secret source file doesn't exist, rather than just logging a warning and creating a folder for the bind mount.

[ndeloof]

indeed, on OSX I get

 ✘ Container truc-sample-1  Error response from daemon: invalid mount config for type "bind": bind source path does not exist...                           0.0s 
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /host_mnt/Users/nicolas/xx/sample.secret

it seems engine doesn't respect the CreateMountpoint API parameter if you don't get such an error reported.
Let me try to reproduce on my Windows machine

[micdet-delen]

I seem to have the same issue with bind mounts (on Windows with WSL integration).

Was running with create_host_path: false. And got build error for every path not existing on the host. As I intended and wanted.

Updated to Docker Desktop 4.49.0 (Compose: v2.40.2-desktop.1) and create_host_path: false stopped working. Ie: the build works and the file is created again as a directory when composing.

Regression in Docker Desktop 4.49.0/Compose: v2.40.2-desktop.1 ? I will try to have Docker Desktop downgraded to previous version and test again (company laptop, so not that simple).

[ndeloof]

I created docker/cli#6573 to make it easier confirm such regression is a docker engine or compose issue in the future

[ndeloof]

I was able to reproduce this issue on Windows with Docker Desktop 4.49, checked that compose sets the expected CreateHostPath flag to false setting container mounts. Sounds like a Docker Desktop issue

[ndeloof]

I can confirm a bug exists on Docker Desktop Windows (not compose fault 😇):

C:\Users\nicolas\test>dir

06/11/2025  10:48    <DIR>          .
06/11/2025  10:05    <DIR>          ..
04/11/2025  16:30               157 compose.yaml

C:\Users\nicolas\test>docker run --mount type=bind,source=C:\Users\nicolas\test\sample.secret,target=/foo alpine ls /foo

# expected an error to be reported, as mount API does **not** create host path until explicitly requested to 
C:\Users\nicolas\test>dir

06/11/2025  10:48    <DIR>          .
06/11/2025  10:05    <DIR>          ..
06/11/2025  10:05    <DIR>          sample.secret
04/11/2025  16:30               157 compose.yaml

[micdet-delen]

Hi @ndeloof , can you elaborate a bit more on your PR docker/cli#6573 ?

This is what I found up to now: I tested the create_host_path: false behaviour on bind mounts in all the following versions (I was not able to go further back due to time constraints):

• Docker 4.46 / Engine 28.4.0 / Compose: v2.39.2
• Docker 4.47 / Engine 28.4.0 / Compose: v2.39.4
• Docker 4.48 / Engine 28.5.1 / Compose v2.40.0
• Docker 4.49 / Engine 28.5.1 / Compose v2.40.2

And in every one of those a new directory is created when the host path is missing. So even when create_host_path is set to false! (to be clear: I only tested this for bind mounts).

@ndeloof : you confirm this behaviour that I'm seeing and you also agree that this is a bug?

I found this interesting topic that was started in 2016:

docker-compose up creates directories on host system · Issue #2781 · docker/compose

Looks like there was a recent change in that area (PR: #12734), which reverted some code-paths to use the old API; that sounds like a regression at least (as the advanced syntax should default to create_host_path: false)?

But it got a resolution in May this year(!):

In the end, this was resolved by updating the compose spec docs to match the current logic in compose-spec/compose-spec#599

I think the final resolution is wrong, because

1. Now the default behaviour (when not specifiying create_host_path) suddenly changed from erroring out when missing paths to creating directories (and possibly leading to all sorts of unwanted behaviour in the container).
2. I think also that now there might be an error in the code, that leads to no longer registering an expliciet create_host_path: false setting? So you always get the new default behaviour now.

[ndeloof]

Agree this is a bug. The bug is not in compose, which use mount API for secrets, but in Docker Desktop as demonstrated by reproduction example #13310 (comment)

About compose-spec/compose-spec#599, compose volume long syntax allows to set create_host_path: false - this forces Compose to use the mount API (same as docker run --mount ...). By default Compose uses bind API (same as docker run -v source:target) for backward compatibility.
the main difference between those is that bind create host path, while mount doesn't (until explicitly requested to).
This documentation fix is a clarification default value is create_host_path: true

#2781 is a very old issue (reported on Docker Compose v1, python) which basically was addressed by the introduction of the mount API and support for create_host_path: false. This isn't related to the current issue

[ndeloof]

note: Internally tracked as https://docker.atlassian.net/browse/PRS-891

[micdet-delen]

Ok, so both issues, ie the secrets files issue from @filimonic and my issue with bind mounts have the same root cause? And will be fixed in the Docker project?

Do you have any idea in what version of Docker was this bug introduced? I tested back to version 4.46 but was still hitting it.

[micdet-delen]

I also see this behaviour from compose, I'm not 100% sure how to interpret it, I'm not familiar enough with Docker and compose, but I do find it strange:

compose file (partial):

      - type: bind
        source: ~/.my_container_bash_history
        target: /home/vscode/.bash_history
        bind:
          create_host_path: false

output of docker-compose config:

      - type: bind
        source: /home/myuser/.my_container_bash_history
        target: /home/vscode/.bash_history
        bind: {}

output of docker-compose config --no-interpolate:

      - bind:
          create_host_path: false
        source: /home/myuser/.my_container_bash_history
        target: /home/vscode/.bash_history
        type: bind

This might be because I have a lack of understanding about what "--no-interpolate" does... So bear with me...

Why does compose config show bind: {} ? I feel like it does that because it thinks the bind option has been set to the default values and thus leaves it "empty" ? Whilst it is not ofcourse, create_host_path: false is not the default.
And --no-interpolate shows me the actual configured value without making assumptions about default values?

[ndeloof]

sounds like a (distinct) bug. default value bing true make the marshaller ignore false value - this was a design mistake of mine 😅