storage.Directory.export_to_tar: default mtime=utils._magic_timestamp

Change the default value of mtime when doing export_to_tar() from '0' to _magic_timestamp. This avoids problems in other software, which assume that an mtime of '0' means the file does not exist.

There is more than one example where files with an mtime of zero are treated as non-existant by other software. e.g. ninja and Template Toolkit.

The OSTree project also express an intention to move from an mtime of 0 to an mtime of 1:

For this reason, OSTree acts as though all timestamps are set to time_t 0, so that comparisons will be considered up-to-date. Note that for a few releases, OSTree used 1 to fix warnings such as GNU Tar emitting "implausibly old time stamp" with 0; however, until we have a mechanism to transition cleanly to 1, for compatibilty OSTree is reverted to use zero again.

From the comments on export_to_tar(), the motivation for having an mtime of 0 was to have reproducible results, rather than it specifically being the value 0.

Additionally, the reproducible builds project has a page on archives; it mentions the benefits of setting all the files to have the same mtime, or clamping the mtime. It makes no mention of a motivation for the mtime to be specifically 0.

Fixes #914 (closed)

Makes sense to me, we use a magic timestamp instead of 0 when staging things in the sandbox for the same reasons (see utils._set_deterministic_mtime())

Cool, and after talking with @danielsilverstone-ct, it seems like utils._magic_timestamp would be better than 1. If users want to honour SOURCE_DATE_EPOCH, then perhaps in later work we can add a --mtime argument for them to make that happen.

marked as a Work In Progress

changed title from storage.Directory.export_to_tar: default mtime=1 to WIP: storage.Directory.export_to_tar: default mtime=utils._set_deterministic_mtime

changed the description

changed title from WIP: storage.Directory.export_to_tar: default mtime=utils._set_deterministic_mtime to WIP: storage.Directory.export_to_tar: default mtime=utils._magic_timestamp

added 1 commit

• ed49da18 - storage.Directory.export_to_tar: mtime=_magic...

Compare with previous version

unmarked as a Work In Progress

added 13 commits

• ed49da18...51dae747 - 12 commits from branch master
• 55a2e5bc - storage.Directory.export_to_tar: mtime=_magic...

Compare with previous version

changed the description

@tristanvb - I think you were kindly saying this was good to merge but I'm not certain, is it good now with the _magic_timestamp change? Cheers!

After yesterdays discussions this looks as expected for me, including the test! Definitely think adding the ability to set SOURCE_DATE_EPOCH here is something worth coming back to in the future.

approved this merge request

Thanks! Seems like there's enough support for this then, will merge.

enabled an automatic merge when the pipeline for 55a2e5bc succeeds

approved this merge request

I think you were kindly saying this was good to merge but I'm not certain, is it good now with the _magic_timestamp change?

Sure !

I was just making a reference to the magic timestamp as an example of why we do something similar in another place, but I suppose it doesn't hurt to use the same one :)

added 3 commits

• 55a2e5bc...14176e51 - 2 commits from branch master
• 39febfda - storage.Directory.export_to_tar: mtime=_magic...

Compare with previous version

enabled an automatic merge when the pipeline for 39febfda succeeds

merged