GitLab Next

What does this MR do?

Adds a rake task that can list, or delete move to lost-and-found, orphaned local project upload files.

Are there points in the code the reviewer needs to double check?

Why was this MR needed?

It is necessary for https://gitlab.com/gitlab-org/gitlab-ce/issues/49630 and https://gitlab.com/gitlab-org/gitlab-ee/issues/6012

Screenshots (if relevant)

Example output:

$ sudo gitlab-rake gitlab:cleanup:project_uploads
I, [2018-07-27T12:08:28.671559 #89817]  INFO -- : Looking for orphaned project uploads to clean up. Dry run...
E, [2018-07-27T12:08:28.689869 #89817] ERROR -- : Skipping... Unable to parse project upload path: "/opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out"
I, [2018-07-27T12:08:28.754259 #89817]  INFO -- : Found correct path! /opt/gitlab/embedded/service/gitlab-rails/public/uploads/qux/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt
I, [2018-07-27T12:08:28.755624 #89817]  INFO -- : Can be moved to parent: /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt
I, [2018-07-27T12:08:28.760257 #89817]  INFO -- : Can be moved to lost and found: /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png
I, [2018-07-27T12:08:28.764470 #89817]  INFO -- : To cleanup these files run this command with DRY_RUN=false

$ sudo gitlab-rake gitlab:cleanup:project_uploads DRY_RUN=false
I, [2018-07-27T12:09:24.944414 #89936]  INFO -- : Looking for orphaned project uploads to clean up...
E, [2018-07-27T12:09:24.962321 #89936] ERROR -- : Skipping... Unable to parse project upload path: "/opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out"
I, [2018-07-27T12:08:28.754259 #89817]  INFO -- : Found correct path! /opt/gitlab/embedded/service/gitlab-rails/public/uploads/qux/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt
I, [2018-07-27T12:08:28.755624 #89817]  INFO -- : Moved to parent: /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt
I, [2018-07-27T12:08:28.760257 #89817]  INFO -- : Moved to lost and found: /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png

Does this MR meet the acceptance criteria?

• Changelog entry added, if necessary
• Documentation created/updated
• Tests added for this feature/bug
• Conforms to the code review guidelines
  • Has been reviewed by a Backend maintainer
• Conforms to the merge request performance guidelines
• Conforms to the style guides
• Conforms to the database guides
• If you have multiple commits, please combine them into a few logically organized commits by squashing them
• End-to-end tests pass (package-and-qa manual pipeline job)

What are the relevant issue numbers?

https://gitlab.com/gitlab-org/gitlab-ce/issues/49630 and https://gitlab.com/gitlab-org/gitlab-ee/issues/6012