Migrate GitLab Pages to zip storage in the background

What does this MR do?

Currently, the only way to migrate pages to zip storage is to manually run the rake task:

gitlab-rake gitlab:pages:migrate_legacy_storage

It requires human intervention on migration and can take a long time to complete. (it took about a week on gitlab.com).

We can't remove this rake task because it allows users to control the process and fix some errors:

• it prints all errors by default, such as invalid symlinks and projects marked as deployed by absent on pages storage
• it also allows for fix these errors by ignoring invalid symlinks and marking projects as not deployed
• and it allows users to see the progress/check if everything has been migrated

However, making every instance admin run this task when upgrading to 14.0 may create too much work for them. So we want to run the same migration code as the background migration. This way we will migrate the data for most of the users without their intervention.

It still will be recommended to run the rake task and make sure everything has been migrated properly. But in most cases everything should be migrated automatically and even if users forgot to run the rake task everything should be fine.

We aim to merge this prior to 14.0, so we have more time to test it, and also for some users all the data will be migrated before they start upgrading to 14.0.

Currently, the background migration code uses services code, and these services rely on some models code. This goes against our guidelines https://docs.gitlab.com/ee/development/background_migrations.html#isolation.

But isolating this migration will require copying significant part of the code:

1. models (I attempted this in the original version)

+    module Routable
+      def build_full_path
+        if parent && path
+          parent.build_full_path + '/' + path
+        else
+          path
+        end
+      end
+    end
+
+    # Namespace
+    class Namespace < ActiveRecord::Base
+      self.table_name = 'namespaces'
+      self.inheritance_column = :_type_disabled
+
+      include Routable
+
+      belongs_to :parent, class_name: "Namespace"
+    end
+
+    # copy class with only required code
+    class Project < ActiveRecord::Base
+      self.table_name = 'projects'
+      self.inheritance_column = :_type_disabled
+
+      include Routable
+
+      belongs_to :namespace
+      alias_method :parent, :namespace
+      alias_attribute :parent_id, :namespace_id
+
+      has_one :pages_metadatum, class_name: 'ProjectPagesMetadatum', inverse_of: :project
+
+      def pages_path
+        File.join(Settings.pages.path, build_full_path)
+      end
+    end
+
+    # copy class with only required code
+    class ProjectPagesMetadatum < ActiveRecord::Base
+      self.table_name = 'project_pages_metadata'
+      self.inheritance_column = :_type_disabled
+
+      include EachBatch
+
+      belongs_to :project, inverse_of: :pages_metadatum, class_name: 'Project'
+      belongs_to :pages_deployment, class_name: 'PagesDeployment'
+
+      scope :deployed, -> { where(deployed: true) }
+      scope :only_on_legacy_storage, -> { deployed.where(pages_deployment: nil) }
+      scope :with_project_route_and_deployment, -> { preload(:pages_deployment, project: [:namespace, :route]) }
+    end
+
+    # copy class with only required code
+    class PagesDeployment < ActiveRecord::Base
+      self.table_name = 'pages_deployments'
+      self.inheritance_column = :_type_disabled
+
+      include FileStoreMounter
+
+      attribute :file_store, :integer, default: -> { ::Pages::DeploymentUploader.default_store }
+
+      belongs_to :project, class_name: 'Project'
+
+      validates :file, presence: true
+      validates :file_store, presence: true, inclusion: { in: ObjectStorage::SUPPORTED_STORES }
+      validates :size, presence: true, numericality: { greater_than: 0, only_integer: true }
+      validates :file_count, presence: true, numericality: { greater_than_or_equal_to: 0, only_integer: true }
+      validates :file_sha256, presence: true
+
+      before_validation :set_size, if: :file_changed?
+
+      # Shall we copy all the uploader code migration as well?
+      mount_file_store_uploader ::Pages::DeploymentUploader
+
+      private
+
+      def set_size
+        self.size = file.size
+      end
+    end

2. Also note, that PagesDeployments mounts ::Pages::DeploymentUploader, which in turn is based on GitlabUploader, ObjectStorage::Concern and requires Gitlab.config.pages.
3. And we use services currently located under app/services/pages/

I can try to move most of this under the lib/gitlab/background_migration/migrate_pages_to_zip_storage/ dir and reuse this code in the rake task, but I don't think it will make the migration more reliable.

Also the current approach was suggested in !54578 (comment 513143758)

Migration output

./bin/rails db:migrate:redo VERSION=20210302150310                                                                                                                 
== 20210302150310 ScheduleMigratePagesToZipStorage: reverting =================
== 20210302150310 ScheduleMigratePagesToZipStorage: reverted (0.0000s) ========

== 20210302150310 ScheduleMigratePagesToZipStorage: migrating =================
-- Scheduled 1 MigratePagesToZipStorage jobs with a maximum of 100 records per batch and an interval of 120 seconds.

The migration is expected to take at least 120 seconds. Expect all jobs to have completed after 2021-03-29 13:50:04 UTC."
== 20210302150310 ScheduleMigratePagesToZipStorage: migrated (0.0521s) ========

Database queries for the schedulling migration

SELECT "project_pages_metadata"."project_id"
FROM "project_pages_metadata"
WHERE "project_pages_metadata"."deployed" = TRUE
  AND "project_pages_metadata"."pages_deployment_id" IS NULL
ORDER BY "project_pages_metadata"."project_id" ASC
LIMIT 1

https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/3119/commands/10272

SELECT "project_pages_metadata"."project_id"
FROM "project_pages_metadata"
WHERE "project_pages_metadata"."deployed" = TRUE
  AND "project_pages_metadata"."pages_deployment_id" IS NULL
  AND "project_pages_metadata"."project_id" >= 3707468
ORDER BY "project_pages_metadata"."project_id" ASC
LIMIT 1
OFFSET 100;

https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/3119/commands/10273

SELECT MIN(project_id),
       MAX(project_id)
FROM "project_pages_metadata"
WHERE "project_pages_metadata"."deployed" = TRUE
  AND "project_pages_metadata"."pages_deployment_id" IS NULL
  AND "project_pages_metadata"."project_id" >= 3707468;

https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/3119/commands/10274

Database query for selecting batch(used in background migration)

SELECT "project_pages_metadata".*
FROM "project_pages_metadata"
WHERE "project_pages_metadata"."deployed" = TRUE
  AND "project_pages_metadata"."pages_deployment_id" IS NULL
  AND "project_pages_metadata"."project_id" BETWEEN 3707468 AND 24766231;

https://postgres.ai/console/gitlab/gitlab-production-tunnel/sessions/3518/commands/11644

Time estimation

We already migrated almost all of the gitlab.com(there are a few projects which failed to be deleted and marked as deployed), so in our own production this migration will be finished with only one iteration required, so it will take only a few minutes.

It took us about a week to run the migration by hand using 3 threads. So I'd say for instances with similar size this migration can take up to a months. (I don't think such instances exist though)

Rake migration updated output

vlad @ gdk1 ➜  gitlab git:(322001-poc-for-migrating-pages-to-zip-storage-in-the-background)  ./bin/rake gitlab:pages:migrate_legacy_storage                                                                                               
I, [2021-04-15T14:45:26.167797 #1818942]  INFO -- : Starting to migrate legacy pages storage to zip deployments
I, [2021-04-15T14:45:26.222246 #1818942]  INFO -- : {:message=>"Pages legacy storage migration: Waiting for threads to finish..."}
E, [2021-04-15T14:45:26.859070 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2", :project_id=>3, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2", :duration=>0.11}
E, [2021-04-15T14:45:26.862268 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/demo-prepare", :project_id=>21, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/demo-prepare", :duration=>0.06}
E, [2021-04-15T14:45:26.862436 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/norbert/underscore", :project_id=>11, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/norbert/underscore", :duration=>0.12}
E, [2021-04-15T14:45:26.869745 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat", :project_id=>4, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat", :duration=>0.01}
E, [2021-04-15T14:45:26.872095 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test", :project_id=>23, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test", :duration=>0.01}
E, [2021-04-15T14:45:26.875534 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2", :project_id=>12, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2", :duration=>0.01}
E, [2021-04-15T14:45:26.882527 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore", :project_id=>5, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore", :duration=>0.01}
E, [2021-04-15T14:45:26.883321 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat", :project_id=>13, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat", :duration=>0.01}
E, [2021-04-15T14:45:26.886693 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2", :project_id=>24, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2", :duration=>0.01}
E, [2021-04-15T14:45:26.892480 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/flightjs/flight", :project_id=>6, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/flightjs/flight", :duration=>0.01}
E, [2021-04-15T14:45:26.895540 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat", :project_id=>14, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat", :duration=>0.01}
E, [2021-04-15T14:45:26.899355 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3", :project_id=>25, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3", :duration=>0.01}
E, [2021-04-15T14:45:26.905503 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js", :project_id=>7, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js", :duration=>0.01}
E, [2021-04-15T14:45:26.905667 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4", :project_id=>26, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4", :duration=>0.0}
E, [2021-04-15T14:45:26.908501 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat", :project_id=>15, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat", :duration=>0.01}
E, [2021-04-15T14:45:26.914845 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5", :project_id=>27, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5", :duration=>0.0}
E, [2021-04-15T14:45:26.918058 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate", :project_id=>8, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate", :duration=>0.01}
E, [2021-04-15T14:45:26.923015 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore", :project_id=>16, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore", :duration=>0.01}
E, [2021-04-15T14:45:26.924688 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/tony/lab-coat", :project_id=>9, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/tony/lab-coat", :duration=>0.0}
E, [2021-04-15T14:45:26.928938 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test", :project_id=>28, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test", :duration=>0.01}
E, [2021-04-15T14:45:26.935466 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore", :project_id=>17, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore", :duration=>0.01}
E, [2021-04-15T14:45:26.941866 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/alecia/wget2", :project_id=>10, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/alecia/wget2", :duration=>0.01}
E, [2021-04-15T14:45:26.942736 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks", :project_id=>30, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks", :duration=>0.01}
E, [2021-04-15T14:45:26.946356 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat", :project_id=>18, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat", :duration=>0.01}
E, [2021-04-15T14:45:26.953684 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell", :project_id=>2, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell", :duration=>0.01}
E, [2021-04-15T14:45:26.954405 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages-test", :project_id=>22, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/pages-test", :duration=>0.01}
E, [2021-04-15T14:45:26.955766 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-environments", :project_id=>19, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/prepare-environments", :duration=>0.01}
E, [2021-04-15T14:45:26.977514 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-demo", :project_id=>20, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/prepare-demo", :duration=>0.02}
I, [2021-04-15T14:45:26.977960 #1818942]  INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>0, :errored=>28}
I, [2021-04-15T14:45:27.112577 #1818942]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: ", :project_id=>1, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test", :duration=>0.16}
I, [2021-04-15T14:45:27.112742 #1818942]  INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>1, :errored=>28}
I, [2021-04-15T14:45:27.116354 #1818942]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: ", :project_id=>29, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/os-test", :duration=>0.16}
I, [2021-04-15T14:45:27.116664 #1818942]  INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>2, :errored=>28}
E, [2021-04-15T14:45:27.146838 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks", :project_id=>31, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks", :duration=>0.0}
E, [2021-04-15T14:45:27.149848 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages", :project_id=>34, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/pages", :duration=>0.0}
E, [2021-04-15T14:45:27.152698 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/new-pages", :project_id=>35, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/new-pages", :duration=>0.0}
E, [2021-04-15T14:45:27.155623 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io", :project_id=>33, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io", :duration=>0.0}
E, [2021-04-15T14:45:27.158060 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test", :project_id=>32, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test", :duration=>0.0}
I, [2021-04-15T14:45:27.197230 #1818942]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: ", :project_id=>36, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/new-new-pages", :duration=>0.04}
I, [2021-04-15T14:45:27.197382 #1818942]  INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>3, :errored=>33}
I, [2021-04-15T14:45:27.197970 #1818942]  INFO -- : A total of 36 projects were processed.
I, [2021-04-15T14:45:27.198056 #1818942]  INFO -- : - The 3 projects migrated successfully
I, [2021-04-15T14:45:27.198111 #1818942]  INFO -- : - The 33 projects failed to be migrated
vlad @ gdk1 ➜  gitlab git:(322001-poc-for-migrating-pages-to-zip-storage-in-the-background) ✗  PAGES_MIGRATION_IGNORE_INVALID_ENTRIES=true PAGES_MIGRATION_MARK_PROJECTS_AS_NOT_DEPLOYED=true ./bin/rake gitlab:pages:migrate_legacy_storage
I, [2021-04-15T14:46:06.655333 #1819371]  INFO -- : Starting to migrate legacy pages storage to zip deployments
I, [2021-04-15T14:46:06.713816 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: Waiting for threads to finish..."}
I, [2021-04-15T14:46:07.307180 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test? Marked project as not deployed", :project_id=>23, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test", :duration=>0.07}
I, [2021-04-15T14:46:07.308834 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/norbert/underscore? Marked project as not deployed", :project_id=>11, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/norbert/underscore", :duration=>0.09}
I, [2021-04-15T14:46:07.314805 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2? Marked project as not deployed", :project_id=>12, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2", :duration=>0.08}
I, [2021-04-15T14:46:07.324846 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat? Marked project as not deployed", :project_id=>13, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat", :duration=>0.01}
I, [2021-04-15T14:46:07.325667 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2? Marked project as not deployed", :project_id=>24, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2", :duration=>0.01}
I, [2021-04-15T14:46:07.327178 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2? Marked project as not deployed", :project_id=>3, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2", :duration=>0.02}
I, [2021-04-15T14:46:07.337607 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat? Marked project as not deployed", :project_id=>14, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat", :duration=>0.01}
I, [2021-04-15T14:46:07.342196 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3? Marked project as not deployed", :project_id=>25, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3", :duration=>0.01}
I, [2021-04-15T14:46:07.345880 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat? Marked project as not deployed", :project_id=>4, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat", :duration=>0.02}
I, [2021-04-15T14:46:07.353219 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat? Marked project as not deployed", :project_id=>15, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat", :duration=>0.01}
I, [2021-04-15T14:46:07.365904 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4? Marked project as not deployed", :project_id=>26, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4", :duration=>0.02}
I, [2021-04-15T14:46:07.371397 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore? Marked project as not deployed", :project_id=>5, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore", :duration=>0.02}
I, [2021-04-15T14:46:07.372261 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore? Marked project as not deployed", :project_id=>16, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore", :duration=>0.02}
I, [2021-04-15T14:46:07.390802 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5? Marked project as not deployed", :project_id=>27, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5", :duration=>0.02}
I, [2021-04-15T14:46:07.392840 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore? Marked project as not deployed", :project_id=>17, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore", :duration=>0.01}
I, [2021-04-15T14:46:07.399543 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/flightjs/flight? Marked project as not deployed", :project_id=>6, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/flightjs/flight", :duration=>0.02}
I, [2021-04-15T14:46:07.409148 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test? Marked project as not deployed", :project_id=>28, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test", :duration=>0.02}
I, [2021-04-15T14:46:07.410922 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat? Marked project as not deployed", :project_id=>18, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat", :duration=>0.02}
I, [2021-04-15T14:46:07.421209 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js? Marked project as not deployed", :project_id=>7, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js", :duration=>0.02}
I, [2021-04-15T14:46:07.429162 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/demo-prepare? Marked project as not deployed", :project_id=>21, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/demo-prepare", :duration=>0.02}
I, [2021-04-15T14:46:07.432642 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks? Marked project as not deployed", :project_id=>30, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks", :duration=>0.02}
I, [2021-04-15T14:46:07.448571 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate? Marked project as not deployed", :project_id=>8, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate", :duration=>0.02}
I, [2021-04-15T14:46:07.455081 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-environments? Marked project as not deployed", :project_id=>19, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/prepare-environments", :duration=>0.02}
I, [2021-04-15T14:46:07.460717 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages-test? Marked project as not deployed", :project_id=>22, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/pages-test", :duration=>0.02}
I, [2021-04-15T14:46:07.475750 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/tony/lab-coat? Marked project as not deployed", :project_id=>9, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/tony/lab-coat", :duration=>0.02}
I, [2021-04-15T14:46:07.482210 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-demo? Marked project as not deployed", :project_id=>20, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/prepare-demo", :duration=>0.02}
I, [2021-04-15T14:46:07.484969 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>26, :errored=>0}
I, [2021-04-15T14:46:07.485870 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks? Marked project as not deployed", :project_id=>31, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks", :duration=>0.02}
I, [2021-04-15T14:46:07.496882 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/alecia/wget2? Marked project as not deployed", :project_id=>10, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/alecia/wget2", :duration=>0.02}
I, [2021-04-15T14:46:07.513568 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test? Marked project as not deployed", :project_id=>32, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test", :duration=>0.01}
I, [2021-04-15T14:46:07.513888 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>29, :errored=>0}
I, [2021-04-15T14:46:07.523019 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell? Marked project as not deployed", :project_id=>2, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell", :duration=>0.02}
I, [2021-04-15T14:46:07.523416 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>30, :errored=>0}
I, [2021-04-15T14:46:07.579295 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages? Marked project as not deployed", :project_id=>34, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/pages", :duration=>0.01}
I, [2021-04-15T14:46:07.586227 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/new-pages? Marked project as not deployed", :project_id=>35, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/new-pages", :duration=>0.01}
I, [2021-04-15T14:46:07.594834 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io? Marked project as not deployed", :project_id=>33, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io", :duration=>0.01}
I, [2021-04-15T14:46:07.595163 #1819371]  INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>33, :errored=>0}
I, [2021-04-15T14:46:07.595741 #1819371]  INFO -- : A total of 33 projects were processed.
I, [2021-04-15T14:46:07.595974 #1819371]  INFO -- : - The 33 projects migrated successfully
I, [2021-04-15T14:46:07.596187 #1819371]  INFO -- : - The 0 projects failed to be migrated

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Background migration logs

(I changed batch size to 20, and time to 2 minutes to speed up tests locally)

vlad @ gdk1 ➜  gitlab git:(322001-poc-for-migrating-pages-to-zip-storage-in-the-background) ✗  tail -F log/application_json.log | grep "legacy"            
{"severity":"ERROR","time":"2021-04-15T13:03:11.071Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/norbert/underscore","project_id":11,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/norbert/underscore","duration":0.02}
{"severity":"ERROR","time":"2021-04-15T13:03:11.074Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2","project_id":12,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.077Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2","project_id":3,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.080Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat","project_id":13,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.085Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat","project_id":14,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.090Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat","project_id":4,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.094Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat","project_id":15,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.097Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore","project_id":5,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.100Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore","project_id":16,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.103Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/flightjs/flight","project_id":6,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/flightjs/flight","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.105Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore","project_id":17,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.108Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat","project_id":18,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.111Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js","project_id":7,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.114Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate","project_id":8,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.117Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-environments","project_id":19,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/prepare-environments","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.120Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/tony/lab-coat","project_id":9,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/tony/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.122Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-demo","project_id":20,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/prepare-demo","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.126Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/alecia/wget2","project_id":10,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/alecia/wget2","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.128Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell","project_id":2,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell","duration":0.0}
{"severity":"INFO","time":"2021-04-15T13:03:11.246Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project migrated: ","project_id":1,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test","duration":0.12}
{"severity":"INFO","time":"2021-04-15T13:03:11.246Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: batch processed","migrated":1,"errored":19}
{"severity":"ERROR","time":"2021-04-15T13:05:10.402Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test","project_id":23,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.408Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2","project_id":24,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.413Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3","project_id":25,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.418Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4","project_id":26,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.421Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5","project_id":27,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.425Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test","project_id":28,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.428Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks","project_id":30,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.431Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/demo-prepare","project_id":21,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/demo-prepare","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.433Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages-test","project_id":22,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/pages-test","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.435Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks","project_id":31,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.437Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test","project_id":32,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.439Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages","project_id":34,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/pages","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.441Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/new-pages","project_id":35,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/new-pages","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.443Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io","project_id":33,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io","duration":0.0}
{"severity":"INFO","time":"2021-04-15T13:05:10.489Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project migrated: ","project_id":29,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/os-test","duration":0.04}
{"severity":"INFO","time":"2021-04-15T13:05:10.529Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project migrated: ","project_id":36,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/new-new-pages","duration":0.04}
{"severity":"INFO","time":"2021-04-15T13:05:10.530Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: batch processed","migrated":2,"errored":14}

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team

Related to #322001 (closed)