Repository navigation
PowerShell provisioner: retrying the env-var upload sends an empty file, so scripts run without environment_vars #13734
Description
Community Note
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Overview of the Issue
Sometimes, one script in a long powershell provisioner step runs with none of its environment_vars set. Scripts before it in the same step got their variables, and so do scripts after it. The script then fails at its first use of a variable, with a misleading error such as:
Cannot bind argument to parameter 'Path' because it is null.
We have seen this 4 times in 120 days of Windows Server 2022/2025 Azure image builds over WinRM. A different script failed each time. We have never seen it on Linux builds that use the shell provisioner.
Root cause
Before each script, uploadEnvVars in provisioner/powershell/provisioner.go writes the flattened env vars to remote_env_var_path (default C:/Windows/Temp/packer-ps-env-vars-<uuid>.ps1). The script then dot-sources that file. The upload runs inside retry.Config{...}.Run, but the strings.Reader is created once, outside the retry closure:
envVarReader := strings.NewReader(flattenedEnvVars)
err = retry.Config{StartTimeout: p.config.StartRetryTimeout}.Run(ctx, func(context.Context) error {
if err := p.communicator.Upload(p.config.RemoteEnvVarPath, envVarReader, nil); err != nil {If an attempt fails after the communicator has read some or all of the reader, the retry reuses the same reader. That reader is now partly or fully consumed, so the retry uploads a truncated or empty file and returns success.
The WinRM communicator makes this likely. winrmcp reads the whole input into a temp file first, then runs two more WinRM shells: one to restore the file to its destination and one to remove the temp file. A transient failure in either of those steps fails the Upload after the reader is already at EOF. On the retry, winrmcp reads 0 bytes, writes no temp file, and its restore script falls into echo $null > $dest_file_path. That creates an empty destination file and exits 0. If the earlier attempt only failed while removing its temp file, the file it had already written correctly is overwritten with the empty one.
The script upload in the same provisioner does not have this problem: it calls f.Seek(0, 0) at the start of every retry attempt. The shell provisioner also rewinds its env var file inside its retry loop, which explains why Linux builds are not affected.
Why it is hard to diagnose
- The retried upload reports success, so nothing fails at upload time.
- Dot-sourcing an empty
.ps1file is valid PowerShell, so nothing fails there either. - The script then fails later, at whatever line first uses a variable. The error points at the user's script, not at Packer.
- The only sign is a
Retryable error: Error uploading ps script containing env vars: ...line in thePACKER_LOG=1output just before the failing script.
Affected components: PowerShell provisioner (provisioner/powershell), both the non-elevated and the elevated_user paths, since both go through prepareEnvVars → uploadEnvVars. Communicator: WinRM.
I checked the other uploads on these paths. The elevated wrapper upload in packer-plugin-sdk/guestexec.GenerateElevatedRunner is a single Upload with no retry, and the remote cleanup script creates a new reader inside its retry closure. Neither has this problem.
Reproduction Steps
This is intermittent because it needs a transient WinRM failure during the env var upload. It shows up most under heavy guest load, for example during large installs.
- Build a Windows image over WinRM with a
powershellprovisioner that runs many scripts and setsenvironment_vars. - When a transient WinRM error hits the env var upload after its data has been sent, the log shows
Retryable error: Error uploading ps script containing env vars. The retry then "succeeds". - The next script runs with no environment variables set.
It reproduces deterministically in a unit test, using a mock communicator whose first Upload reads part of the reader and then returns an error. The second Upload receives the unread tail of the text, or an empty string if the first attempt read everything, instead of the full text.
Packer version
1.16.0
Simplified Packer Template
build {
sources = ["source.azure-arm.windows"]
provisioner "powershell" {
environment_vars = [
"INSTALL_ROOT=C:\\install",
"SOME_SETTING=value",
]
scripts = [
"scripts/01.ps1",
"scripts/02.ps1",
# ... many more scripts
"scripts/30.ps1",
]
}
}Operating system and Environment details
- Guests: Windows Server 2022 and Windows Server 2025 (Azure,
azure-armbuilder) - Communicator: WinRM
- Not seen on Linux guests using the
shellprovisioner over SSH
Log Fragments and crash.log files
With PACKER_LOG=1, the failing script comes right after a line like this:
Retryable error: Error uploading ps script containing env vars: <winrm error>
The script then fails with a null-argument error at its first use of an environment variable.
Activity