Verify legacy uploads on Geo secondaries by reading the file
What does this MR do and why?
Geo::UploadReplicator is the legacy replicator that syncs and verifies uploads on every secondary site. Since 18.11, a checksum-adoption shortcut made secondary sites compare the primary site's checksum with itself. This let verification pass for damaged or missing files after the primary site verified an upload. This MR makes parent_upload_verification_succeeded? always return false in the legacy replicator, so secondary sites read the local file again.
Details
The primary site does not change. verify sets the state to verification_started before it calculates the checksum, so the shortcut never applied there.
The partition replicators (e.g Geo::ProjectUploadReplicator) do not change. None of them inherit from Geo::UploadReplicator.
The shared examples split into two groups. 'a blob replicator for upload records' holds the examples common to all upload replicators. 'a blob replicator with upload replicator behavior' includes that group plus the checksum-adoption examples, and now runs only for partition replicators.
The legacy spec uses only the common group. It adds these examples:
calculate_checksumreads the file.- On a secondary site, a damaged file fails the first verification and a reverification with a checksum mismatch.
ok_to_skip_download?still skips the download when the file is present.
This MR and !256554 (merged) change the same shared examples file. Whichever MR merges second must rebase and move the other MR's new checksum-adoption examples into the partition-only group.
This fix is a backport candidate for stable branches that include commit b1c5260d1ba5 (18.11).
References
- Resolves: #630690 (closed)
- Parent epic: gitlab-org#23525
- Related MR: !256554 (merged)
- Related issue: #628305 (closed)
- Issue of the change that caused the bug: #594291 (closed)
Screenshots or screen recordings
This is a backend-only change, so screenshots are not applicable.
How to set up and validate locally
Use a GDK with a Geo primary site and secondary site.
- Add an image to a comment.
- Wait until the secondary site syncs and verifies the upload. Check the Admin area.
- On the secondary site, write different bytes to the file on disk.
- In a Rails console on the secondary site, run
Geo::UploadRegistry.find_by(file_id: <upload ID>).replicator.verify. - Expected result: on
masterthe registry staysverification_succeeded. With this MR, it becomesverification_failed.
bin/rspec ee/spec/replicators/geo/upload_replicator_spec.rb ee/spec/replicators/geo/project_upload_replicator_spec.rbMR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.