Skip to content

Feature: add a rake task to migrate external diffs to object storage

Proposal

In #372324 (closed) we've documented the existing limitation in external diffs migration (from the database to filesystem / object storage).

At the moment the migration is triggered, it will move the external diffs to the configured storage. If that is on the file system, there is no tool to transfer it, other than the mentioned manual steps.

It's important to have this done by something like a rake task to make it less error-prone.

We should consider implementing something similar to the Uploads migration: https://docs.gitlab.com/ee/administration/raketasks/uploads/migrate.html

How to test

  1. Configure gitlab to store external diffs on local disk (gitlab_rails['external_diffs_enabled'] = true).
  2. Generate some diffs.
  3. Configure external diffs object storage (gitlab_rails['external_diffs_object_store_enabled'] = true).
  4. Confirm diffs still on local storage and not on object storage.
  5. Try to access the diffs page for an MR from another rails node. The page will fail to load and a "No such file or directory @ rb_sysopen - /var/opt/gitlab/gitlab-rails/shared/external-diffs/merge_request_diffs/mr-9026/diff-18493" 500 error will be logged.

When a successful migration happens we should not get any error and the diff should be viewable.

Edited by Gabriel Mazetto