Skip to content

Update vfs/zip implementation to ensure minimum range requests for go1.17

Arran Walker requested to merge 625-add-tests-for-stream-reading-zip into master

What does this MR do?

Go 1.17 introduced a bug that, when used alongside a ranged io.readerAt implementation for fetching parts of a zip archive, would result immediately in as many range requests as there are files in the archive upon opening it.

We fixed the bug and merged it upstream (https://go-review.googlesource.com/c/go/+/357489), but it won't be available until Go 1.18. In the mean time, we've created a fork (https://gitlab.com/gitlab-org/golang-archive-zip/) that includes the fix. When we no longer support Go 1.17, this can be removed and archive/zip can be reintroduced.

Along with this change, a test has been added that will fail under such scenarios in the future. Both the test and fix are in two separate commits, so the test can be tested independently of the fix if .tool-versions golang version is updated to golang 1.17.4.

These same fixes have been implemented for workhorse (gitlab!70443 (merged), gitlab!72780 (merged)).

createArchive is used as part of the test because the existing zip files used do not include data descriptors, which is very common in zip files, and is what triggers this bug, but is absent from the zip files currently tested with.

TODO

Closes #625 (closed)

Merge request reports