Skip to content

gitlab-zip-metadata: Ensure 0-byte files have a size field

Stan Hu requested to merge sh-workhorse-include-empty-zip-metadata into master

What does this MR do and why?

As mentioned in https://pkg.go.dev/encoding/json#Marshal:

The "omitempty" option specifies that the field should be omitted from the encoding if the field has an empty value, defined as false, 0, a nil pointer, a nil interface value, and any empty array, slice, map, or string.

As a result, for 0-byte files, the JSON output of the ZIP metadata omits the crc and size fields. Since we always expect this to be generated, drop the omitempty entries from crc, size, and zipped fields.

Relates to #460378 (closed)

How to set up and validate locally

  1. Run this:
make -C workhorse
cp spec/fixtures/ci_build_artifacts.zip /tmp
workhorse/gitlab-zip-metadata /tmp/ci_build_artifacts.zip > /tmp/artifacts.gz
cd /tmp
gunzip artifacts.gz
cat artifacts
  1. You should see the JSON fields with crc and size with size 0:
% cat artifacts
&GitLab Build Artifacts Metadata 0.0.2
{}ci_artifacts.txtK{"modified":1451299530,"mode":"644","crc":280037253,"size":27,"zipped":27}
generated.ymlL{"modified":1580136466,"mode":"644","crc":3678384377,"size":69,"zipped":42}
other_artifacts_0.1.2/A{"modified":1580139332,"mode":"755","crc":0,"size":0,"zipped":0}
other_artifacts_0.1.2/.DS_StoreN{"modified":1580136288,"mode":"644","crc":838385317,"size":6148,"zipped":241}
+other_artifacts_0.1.2/another-subdirectory/A{"modified":1580139332,"mode":"755","crc":0,"size":0,"zipped":0}
4other_artifacts_0.1.2/another-subdirectory/.DS_StoreN{"modified":1580136288,"mode":"644","crc":460543011,"size":6148,"zipped":232}
<other_artifacts_0.1.2/another-subdirectory/banana_sample.gifR{"modified":1447529612,"mode":"644","crc":2003048139,"size":71759,"zipped":71410}
;other_artifacts_0.1.2/another-subdirectory/empty_directory/A{"modified":1451299624,"mode":"755","crc":0,"size":0,"zipped":0}
$other_artifacts_0.1.2/doc_sample.txtO{"modified":1447529612,"mode":"644","crc":1842175207,"size":1314,"zipped":598}
rails_sample.jpgR{"modified":1447529612,"mode":"644","crc":3176234148,"size":35255,"zipped":31916}
tests_encoding/A{"modified":1452590724,"mode":"755","crc":0,"size":0,"zipped":0}
 tests_encoding/non-utf8-dir-≠/A{"modified":1452590942,"mode":"755","crc":0,"size":0,"zipped":0}
.tests_encoding/non-utf8-dir-≠/regular_file_1I{"modified":1452591090,"mode":"644","crc":336514306,"size":7,"zipped":7}
!tests_encoding/utf8 test dir ✓/A{"modified":1452590950,"mode":"755","crc":0,"size":0,"zipped":0}
/tests_encoding/utf8 test dir ✓/regular_file_2J{"modified":1452591076,"mode":"644","crc":1059299009,"size":7,"zipped":7}
Edited by Sylvester Chin

Merge request reports