Repository navigation
Include files under symlinked directories in the sdist - #830
Open
faresrafat3 wants to merge 1 commit into
Open
faresrafat3 wants to merge 1 commit into
faresrafat3 wants to merge 1 commit into
Conversation
A directory symlink nested inside an included directory was silently
dropped from the sdist, because os.walk does not follow directory
symlinks by default:
docs/ <- in sdist include
linked -> ../real_data <- skipped entirely
Walking with followlinks=True picks those files up. Because that alone
would recurse forever on a symlink cycle, the real path of each visited
directory is tracked and repeats are pruned.
Fixes pypa#509.
Author
|
Hi — following up on this PR. It includes files under symlinked directories in the sdist. Let me know if you'd like any changes or if there's anything I can help with. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #509.
A directory symlink nested inside an included directory was silently dropped
from the sdist, so this
pyproject.toml:with this tree:
produced an sdist without
docs/linked/data.txt. Reproduced by building thesdist and listing the tar members — the file is absent.
Root cause
SdistBuilder.apply_includes_excludescollects the contents of each includeddirectory with
os.walk, which does not follow directory symlinks bydefault:
Note that a top-level
include = ["linked_data/"]naming the symlink directlydoes work (the pattern matches the symlink itself), which is why the bug only
shows up for a symlink nested inside a walked directory.
Change
Walk with
followlinks=True. On its own that would recurse forever on asymlink cycle, so the real path of every visited directory is recorded and
repeats are pruned:
Verification
Built the sdist before and after:
docs/linked -> real_datadocs/linked/data.txtpresentdocs/sub/loop -> docs(cycle)The cycle case was measured explicitly because
followlinks=Trueis unsafewithout the guard — with the naive one-line change it produced 45 members with
deeply repeated
loop/sub/loop/sub/...paths before the seen-set was added.Two regression tests added to
tests/test_sdist.py, both drivingapply_includes_excludesdirectly with a fixed file list so they do not dependon git/hg being present: one asserts the symlinked directory's file is
included, the other asserts a symlink cycle terminates with no duplicate
entries.
Full suite:
353 passed, 1 failed— the failure (tests/test_upload.py::test_upload)is pre-existing and reproduces identically on the unmodified tree (it needs the
flitCLI installed, which this environment lacks).