FF_USE_FASTZIP encodes invalid datestamps in gitlab-runner 14.8.1
Summary
After upgrading to gitlab-runner 14.8.1 (from 14.7.0) several of our jobs started failing on both Linux and Windows, when PyInstaller couldn't package files that had date stamps before 1980:
Traceback (most recent call last):
File "C:\builds\software\py-beta-gui\venv\Scripts\build-py-beta-gui-distribution-script.py", line 33, in <module>
sys.exit(load_entry_point('py-beta-gui', 'console_scripts', 'build-py-beta-gui-distribution')())
File "c:\builds\software\py-beta-gui\beta\gui\distribution\build.py", line 18, in main
build('script-interface',
File "C:\builds\software\py-beta-gui\venv\lib\site-packages\beta\common\pyinstaller\build.py", line 152, in build
shutil.make_archive(f'dist/{name}', 'zip', root_dir='dist', base_dir=name)
File "c:\users\containeradministrator\scoop\apps\python38\3.8.10\lib\shutil.py", line 1059, in make_archive
filename = func(base_name, base_dir, **kwargs)
File "c:\users\containeradministrator\scoop\apps\python38\3.8.10\lib\shutil.py", line 956, in _make_zipfile
zf.write(path, path)
File "c:\users\containeradministrator\scoop\apps\python38\3.8.10\lib\zipfile.py", line 1741, in write
zinfo = ZipInfo.from_file(filename, arcname,
File "c:\users\containeradministrator\scoop\apps\python38\3.8.10\lib\zipfile.py", line 539, in from_file
zinfo = cls(arcname, date_time)
File "c:\users\containeradministrator\scoop\apps\python38\3.8.10\lib\zipfile.py", line 362, in __init__
raise ValueError('ZIP does not support timestamps before 1980')
ValueError: ZIP does not support timestamps before 1980
After some investigation, the bad datestamps seem to be present in our cache files. Clearing the cache temporarily fixes the problem, but new caches can still be created with bad timestamps. Either disabling FF_USE_FASTZIP
, or downgrading to gitlab-runner 14.7.1, will provide a more persistent fix.
Some details on one of the bad cache archives; only a few files have bad timestamps, most of the timestamps are correct but a few of them have these bad timestamps with all zeros in the DOS timestamp and no Unix timestamp.
# zipinfo -v py-beta-gui-pip-cache-bad.zip | grep -A20 -B5 fdcb0fb9fc71f8ddecf18a4c989667e496c229788805e26431ed2948
There is no file comment.
Central directory entry #617:
---------------------------
.cache/pip/http/f/d/c/b/0/fdcb0fb9fc71f8ddecf18a4c989667e496c229788805e26431ed2948
offset of local header from start of archive: 149513520
(0000000008E96530h) bytes
file system or operating system of origin: Unix
version of encoding software: 0.0
minimum file system compatibility required: MS-DOS, OS/2 or NT FAT
minimum software version required to extract: 0.0
compression method: deflated
compression sub-type (deflation): normal
file security status: not encrypted
extended local header: no
file last modified on (DOS date/time): 1980 000 0 00:00:00
32-bit CRC value (hex): b37cabd6
compressed size: 165713157 bytes
uncompressed size: 166025134 bytes
length of filename: 82 characters
length of extra field: 15 bytes
length of file comment: 0 characters
disk number on which file begins: disk 1
apparent file type: binary
The same file in a cache generated using gitlab-runner 14.7.1:
# zipinfo -v cache.zip | grep -A20 -B5 fdcb0fb9fc71f8ddecf18a4c989667e496c229788805e26431ed2948
Central directory entry #561:
---------------------------
There are an extra 16 bytes preceding this file.
.cache/pip/http/f/d/c/b/0/fdcb0fb9fc71f8ddecf18a4c989667e496c229788805e26431ed2948
offset of local header from start of archive: 115132594
(0000000006DCC8B2h) bytes
file system or operating system of origin: Unix
version of encoding software: 2.0
minimum file system compatibility required: MS-DOS, OS/2 or NT FAT
minimum software version required to extract: 2.0
compression method: deflated
compression sub-type (deflation): normal
file security status: not encrypted
extended local header: yes
file last modified on (DOS date/time): 2022 Mar 2 18:08:56
file last modified on (UT extra field modtime): 2022 Mar 2 18:08:56 local
file last modified on (UT extra field modtime): 2022 Mar 2 18:08:56 UTC
32-bit CRC value (hex): 1d059b3b
compressed size: 165728814 bytes
uncompressed size: 166025134 bytes
length of filename: 82 characters
length of extra field: 24 bytes
length of file comment: 0 characters
Still working on creating a minimal test case and reporting the bug upstream to fastzip
and/or compress
, but wanted to get this and the workaround documented for GitLab.