Task gitlab:uploads:migrate_to_local:all loses some upload images when configuration is not set correctly
Summary
Upload objects are stored in object storage, then a decision is made to relocate back to local storage.
We have these instructions uploads/migrate.html#migrate-to-local-storage which detail the configuration needed before running this command.
This is very easy to miss.
When you run this rake task, then there is upload image loss. Recovering these means going back to the s3 backups (if being taken) to retrieve the file, as Object storage is not backed up using gitlab-backup.
Steps to reproduce
Configure AWS for Object Storage and don't set
gitlab_rails['uploads_object_store_direct_upload']
gitlab_rails['uploads_object_store_background_upload']these defaults for these are direct_upload: false, background_upload: true
Create an issue within a project and attach some images.
Run Rake task (settings are direct_upload: false, background_upload: true)
- images get pulled down to local
- some uploads deleted from local
- bucket files were deleted resulting in some missing upload files
When following the documentation we get
Run Rake task (settings are direct_upload: false, background_upload: false)
- images get pulled down to local
- bucket files were deleted
Example Project
What is the current bug behavior?
Some images are lost.
What is the expected correct behavior?
No loss of images, Rake tasks detects (myabe) faulty configuration and errors out detailing the mis-configuration.
Relevant logs and/or screenshots
Output of checks
Results of GitLab environment info
GitLabb 13.1.4-ee