Skip to content

Geo: Do not checksum remote stored package files

Problem

Currently, SSF verification will attempt to checksum all package files, including remote stored ones. But it will fail, since we don't have a method to do that at the moment. (Additionally, it is likely to be very bandwidth-intensive to checksum remote stored objects, which is bad for e.g. AWS transfer costs.)

This is the reason for the package file checksum failures on staging.gitlab.com:

image

There are 327 package files, all remotely stored. So checksummable? returns false which makes calculate_checksum return nil which makes the attempted transition to verification_succeeded fail, which then transitions to verification_failed.

Proposal

In the scopes where we get pending or failed records to checksum-- needs_verification_count, verification_pending_batch, and verification_failed_batch:

  • Check if at least one secondary site has sync_object_storage enabled, and if not, then exclude all remote stored files from checksumming via Geo. Edit: We haven't implemented the ability to checksum remote stored objects yet, so for now, just exclude all remote stored files from checksumming via Geo.
Edited by Michael Kozono