Move migrate! method/logic into CarrierWave and ensure data integrity

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Currently, we've pulled a lot of CarrierWave classes/logic for the migrate! method. Also, we manually update file_store out of CarrierWave routine. This gains the complexity of the whole migration processes and risks user data by human errors.

We can contain the whole logic in CarrierWave.

In addition, we can check if the uploaded data is truly same with the source by sending checksum to OS.

Proposal

• Move migrate! method into CarrierWave with file_store updates
• Enable validates_integrity_of validator
• Send content-md5 header when putting object to ObjectStorage (AWS/GCS)
• If no ETag value returned, raise CarrierWave::IntegrityError and surface the error to the comsumer (i.e. GitLab-rails)

Benefit

• Mitigating the whole complexity
• Maintainable and scalable approach
• Resilient
• Tested/Reviewed by contributors and CarrierWave users

Reference

• https://github.com/carrierwaveuploader/carrierwave
• https://github.com/fog/fog-aws
• https://github.com/fog/fog-google
• https://aws.amazon.com/premiumsupport/knowledge-center/data-integrity-s3/
• https://cloud.google.com/storage/docs/hashes-etags