gh-75707: tarfile: Add optional open() argument "mtime"

[ampleyfly]

This makes it possible to create reproducible .tar.gz files without overriding time.time(), by setting the gzip header field mtime to 0.

• Issue: Allow setting timestamp in gzip-compressed tarfiles #75707

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[picnixz]

Please update the documentation of tarfile.open() & co.

[ampleyfly]

Do you think I should do something similar to compresslevel, where it is explicitly only allowed in certain modes (involving creating a gzip archive, in this case), or is it better to simply ignore it if not applicable?

[picnixz]

I'm actually torn between:

• having a single parameter for reproducibility
• allow to set all dynamic parameters separately.

[ampleyfly]

I just realized my test is not failable, since fobj.mtime is not set until decompression has started.
EDIT: Should be fixed now. Also added test for the stream mode.

[maurycy]

@ampleyfly @picnixz

Maybe it's better to just allow setting a timestamp (eg: timestamp or mtime)?

Reproducibility is a strong promise that should be confirmed with tests: is it across the same files? the same user? the same machine? are supported compressions fully deterministic always?

• cpython/Lib/tarfile.py

  Lines 891 to 892 in 96b7a2e

  	uid = 'User ID of the user who originally stored this member.',
  	gid = 'Group ID of the user who originally stored this member.',

• cpython/Lib/tarfile.py

  Lines 901 to 902 in 96b7a2e

  	uname = 'User name.',
  	gname = 'Group name.',

Perhaps we could enable the user to create a reproducible file, and document on how.

[ampleyfly]

@ampleyfly @picnixz

Maybe it's better to just allow setting a timestamp (eg: timestamp or mtime)?

Reproducibility is a strong promise that should be confirmed with tests: is it across the same files? the same user? the same machine? are supported compressions fully deterministic always?

Good points. That was very naïve of me :)
I think the reproducible option would be nice, but I see now that that would require a lot more work.
Is an mtime option a good idea on its own?

[ampleyfly]

The GNU tar documentation has some pointers for making "more reproducible" archives (https://www.gnu.org/software/tar/manual/html_section/Reproducibility.html).
It also claims to show how to accomplish reproducible archives with tar + gzip, noting that gzip is reproducible, with the right options.

[ampleyfly]

This PR is now just adding the mtime option.

[ethanfurman]

Is there a test to ensure that not setting the mtime still functions correctly?

[ampleyfly]

I added tests that use the workaround of temporarily setting time.time to set a known value and check that that value ends up in the mtime field of the gzip header.

[ethanfurman]

That's a fine solution for single-threaded tests, but if test are run in parallel there could eventually be hard-to-reproduce bugs.
Another solution would be to record the time before making the gzip, the time after making the gzip, and then verifying that the gzip header timestamp is somewhere in that range.

[github-actions]

This PR is stale because it has been open for 30 days with no activity.