Repository navigation
Conversation
When the first Range request is refused with 429 or 503, connection 0 retried from the offset saved at Start while Resolve kept downloading. The takeover then read the overlap from the network only to discard it, so a server could send the file up to twice. Retries in resolve-fallback mode now start at the current prefetch position, and connection 0 completes from the prefetch when it already holds the whole body.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related to #1600.
If a connection-limited server answers the first Range request with 429 or 503, connection 0 retries from the offset saved at
Startwhile the Resolve prefetch keeps downloading. When a retry succeeds,commitResolveTakeovercopies the prefetch and reads the overlap from the network again only to discard it.Fix:
commitResolveTakeovernow counts chunk progress fromChunk.Begin.completeFromResolveFallbackand sends no request. With an unknown size this is skipped, because a prefetch stopped byPausealso ends without an error.Traffic for a 4 MiB file in
TestFetcher_RetriedTakeoverDoesNotRefetchPrefix(throttled server, 429 when busy):This doesn't explain the three writers at 0, 25% and 50% in #1600. Zero-filled holes from servers that answer 403 to extra connections are #1592 (PR #1593).
go test -race ./internal/protocol/http/andgo test ./pkg/download/...pass, apart from the existing #1563 race reports fixed in #1606. Go 1.27.3, darwin/arm64, HTTP/1.1 only.