Geo: Enhance "File is not checksummable" error details

Scott Murrayrequested to merge
workloads/c2491bdef1b into master

What does this MR do and why?

  • Enhances the current "File is not checksummable" error messages with specific failure reasons
  • This message is either displayed when the record is excluded from verification, OR when the resource attached to the record doesn't exist
  • This MR updates the error message to let users know which of the above caused the error
  • We also add the expected file path of the missing resource, to help reduce steps when debugging (currently you need to manually query the objects to get the expected file path, then search for them in S3 to confirm they are missing)

References

Relates to issue #556603

How to set up and validate locally

  1. Set up Geo locally https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/72a913f8fbdda624c8f0c9572e3c61d0e14b9e19/doc/howto/geo.md#easy-installation
  2. Trigger the "not checksummable" error by attaching an avatar image to your user profile and physically removing the file:-
# In rails console on primary
upload = Upload.last
file_path = upload.absolute_path
puts "File path: #{file_path}"
  1. Reverify the Upload -
# In rails console on primary
upload = Upload.last
upload.replicator.verify
  1. Query the UploadState record for this upload - which should now have a detailed error:-
> Geo::UploadState.last
=> #<Geo::UploadState:0x00000003151b99d8
 verification_started_at: Thu, 09 Oct 2025 13:07:33.629534000 UTC +00:00,
 verification_retry_at: Thu, 09 Oct 2025 13:07:51.829977000 UTC +00:00,
 verified_at: Thu, 09 Oct 2025 13:07:33.830115000 UTC +00:00,
 upload_id: 65,
 verification_state: 3,
 verification_retry_count: 1,
 verification_checksum: nil,
 verification_failure:
  "Error during verification: File is not checksummable - file at 'gdk-primary/gitlab/public/uploads/-/system/user/avatar/1/avatar.png' does not exist">

MR acceptance checklist

MR Checklist ( @s_murray)
Edited by Scott Murray

Merge request reports