Repository navigation
Retry object deployment after temporary API errors #295 - #510
Merged
Merged
Conversation
Deploy-AdfObjectOnly wraps every Az call in Invoke-AdfWithRetry, which retries time-outs, throttling (429), server errors (5xx) and concurrent operations, waiting 2, 4, 8... seconds. Permanent errors are not retried. New publish option MaxRetries (default 3, 0 = off). Az calls now use -ErrorAction Stop, so a failed deployment stops the publish whatever $ErrorActionPreference is, instead of being skipped and the object marked as deployed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #295. Approach agreed in this comment on #295.
Problem
Every object was deployed with a single API call and no retry, so one temporary time-out failed the whole publish. In the issue's log the request failed after almost exactly 100 seconds, which is the client-side
HttpClienttimeout. Retrying is safe because these calls create or update (HTTPPUT).Changes
Test-AdfTransientError(new, private): decides whether an error is worth retrying, checking inner exceptions too.TaskCanceledExceptionandTimeoutException), HTTP 429 /TooManyRequests, HTTP 5xx, "concurrent operations".BadRequest,NotFound, policy, authorization...).Invoke-AdfWithRetry(new, private): runs the call and retries temporary errors after 2, 4, 8... seconds, with a warning for each retry. Other errors, or the last failed attempt, are passed on.Deploy-AdfObjectOnly: all 10 Az calls are wrapped and called with-ErrorAction Stop. This covers both publish methods:AzResource(New-AzResource, the default and the cmdlet in the issue's log) andAzDataFactory(Set-AzDataFactoryV2*). Global parameters are unchanged.MaxRetries([Int32], default3,0= off). Documented inREADME.md,docs/ADVANCED/CMDLET_REFERENCE.mdanddocs/GUIDE/PUBLISH_OPTIONS.md.Example output with two simulated time-outs:
The Az cmdlets write non-terminating errors. Under the default
$ErrorActionPreference = 'Continue', a failed deployment until now only printed a red error: the publish carried on and the object was marked as deployed (#472 covered terminating errors only). Now any failure that is not retried, or is still failing after the last retry, stops the publish whatever the caller's preference is. This is what #472 intended and has its own changelog line under "Fixed". Pipelines that relied on such errors being ignored will now fail.Tests
New
test/Invoke-AdfWithRetry.Tests.ps1, written before the fix:MaxRetries = 0, the module's ownthrowDeploy-AdfObjectOnlywith each publish method. Before the fix they failed as expected: no retry, and a permanent error did not stop the publish underContinue.Locally with Pester 6.2.0, 82/82 passed under both
StopandContinue: the new file plus the #472 error-handling, publish options, DryRun and trigger tests.Changelog
Two lines under
[Unreleased]: "Added" for the retry andMaxRetries, "Fixed" for the behaviour change.🤖 Generated with Claude Code