Conversation
The progress name of a RUN step is the Dockerfile line passed through the word lexer so that known variables are expanded. The command itself is passed to the shell unchanged, so escapes removed by the lexer only disappear from the name: "RUN echo C:\hello\world" was shown as "RUN echo C:helloworld". Keep escape characters in the name, as is already done for quotes. The lexer still honours them, so "\$FOO" stays unexpanded. This exposed a bug in the lexer's raw-escape mode: inside double quotes, an escape character that escapes nothing was written twice. Signed-off-by: Suhail Hany <suhail9816@gmail.com>
| t.Parallel() | ||
| df := `FROM scratch | ||
| ENV FOO=bar | ||
| RUN echo C:\hello\world\path |
There was a problem hiding this comment.
Doesn't this require an # escape= annotation? IIRC, otherwise \ is still an escape?
There was a problem hiding this comment.
For RUN it isn't: the reference you linked says "regardless of whether the escape parser directive is included in a Dockerfile, escaping is not performed in a RUN command, except at the end of a line."
The shell-form command is passed to the shell as a literal string (handleJSONArgs), so sh -c or cmd /S /C gets echo C:\hello\world\path unchanged. Only the step name goes through the word lexer. That's why the issue shows RUN echo C:helloworldpath as the name, with C:\hello\world\path in the output right below it.
With # escape=` the same happens to backticks: RUN echo C:\hello `$FOO is shown as RUN echo C:\hello $FOO today. I added that case to the test.
Also, I pushed a second commit: my lexer change also affected ${VAR#"..."} patterns, so it's now limited to the case where the quotes are kept.
There was a problem hiding this comment.
For RUN it isn't: the reference you linked says "regardless of whether the escape parser directive is included in a Dockerfile, escaping is not performed in a RUN command, except at the end of a line."
Ah, you're right! Yes, escaping in RUN is handled by whatever is executed by the RUN itself.
(except for the line-continuation at the end, which is part of the Dockerfile parsing)
The previous commit stopped raw-escape mode from writing a literal
escape character inside double quotes twice. That doubling is still
needed when the quotes themselves are dropped, as they are for the
patterns of ${VAR#...}, ${VAR%...} and ${VAR/.../...}: without it,
${FOO#"a\b"} fails with "invalid escape '\b'" instead of matching a
literal "a\b".
Only write the escape character once when the quotes are kept
(RawQuotes), as for RUN step names and heredoc detection.
Signed-off-by: Suhail Hany <suhail9816@gmail.com>
With "# escape=`" the escape character in RUN step names is a backtick, which was dropped from the name the same way as a backslash. Signed-off-by: Suhail Hany <suhail9816@gmail.com>
2 participants
fixes #5250
Build progress drops backslashes from
RUNstep names, which makes Windows paths unreadable.RUN echo C:\hello\worldis shown asRUN echo C:helloworld, even though the shell receives the command unchanged.dispatchRunnow setsRawEscapesnext to the existingRawQuotes. Escapes still apply while lexing, so\$FOOstays unexpanded, but they remain in the name.RawEscapesbug in the lexer. When the quotes are kept, an escape character inside double quotes that escapes nothing was written twice ("C:\hello"became"C:\\hello"). WithoutRawQuotesthe doubling is still needed, since${FOO#"a\b"}relies on it, so that case is unchanged.# escape=`. Unlike fix: dockerfile2llb: handle escaping of backslashes correctly #5269,processCmdEnvis unchanged, so COPY, ADD and WORKDIR names stay the same.Visible change: exec-form lines keep their JSON escapes, so
RUN ["echo", "C:\\path"]is shown as written.For context,
testSecretAsEnvironalready uses forward slashes on Windows "because the Dockerfile parser consumes backslashes as escapes".Testing: the new tests in
dockerfile2llb(step names read fromDockerfile2LLBoutput) andshellfail on master and pass with the fix. The existingshell,parser,instructionsanddockerfile2llbunit tests pass.