Migrate GitLab Pages to zip storage in the background
-
Review changes -
-
Download -
Patches
-
Plain diff
What does this MR do?
Currently, the only way to migrate pages to zip storage is to manually run the rake task:
gitlab-rake gitlab:pages:migrate_legacy_storage
It requires human intervention on migration and can take a long time to complete. (it took about a week on gitlab.com).
We can't remove this rake task because it allows users to control the process and fix some errors:
- it prints all errors by default, such as invalid symlinks and projects marked as deployed by absent on pages storage
- it also allows for fix these errors by ignoring invalid symlinks and marking projects as not deployed
- and it allows users to see the progress/check if everything has been migrated
However, making every instance admin run this task when upgrading to 14.0
may create too much work for them.
So we want to run the same migration code as the background migration.
This way we will migrate the data for most of the users without their intervention.
It still will be recommended to run the rake task and make sure everything has been migrated properly. But in most cases everything should be migrated automatically and even if users forgot to run the rake task everything should be fine.
We aim to merge this prior to 14.0, so we have more time to test it, and also for some users all the data will be migrated before they start upgrading to 14.0.
Currently, the background migration code uses services code, and these services rely on some models code. This goes against our guidelines https://docs.gitlab.com/ee/development/background_migrations.html#isolation.
But isolating this migration will require copying significant part of the code:
- models (I attempted this in the original version)
+ module Routable
+ def build_full_path
+ if parent && path
+ parent.build_full_path + '/' + path
+ else
+ path
+ end
+ end
+ end
+
+ # Namespace
+ class Namespace < ActiveRecord::Base
+ self.table_name = 'namespaces'
+ self.inheritance_column = :_type_disabled
+
+ include Routable
+
+ belongs_to :parent, class_name: "Namespace"
+ end
+
+ # copy class with only required code
+ class Project < ActiveRecord::Base
+ self.table_name = 'projects'
+ self.inheritance_column = :_type_disabled
+
+ include Routable
+
+ belongs_to :namespace
+ alias_method :parent, :namespace
+ alias_attribute :parent_id, :namespace_id
+
+ has_one :pages_metadatum, class_name: 'ProjectPagesMetadatum', inverse_of: :project
+
+ def pages_path
+ File.join(Settings.pages.path, build_full_path)
+ end
+ end
+
+ # copy class with only required code
+ class ProjectPagesMetadatum < ActiveRecord::Base
+ self.table_name = 'project_pages_metadata'
+ self.inheritance_column = :_type_disabled
+
+ include EachBatch
+
+ belongs_to :project, inverse_of: :pages_metadatum, class_name: 'Project'
+ belongs_to :pages_deployment, class_name: 'PagesDeployment'
+
+ scope :deployed, -> { where(deployed: true) }
+ scope :only_on_legacy_storage, -> { deployed.where(pages_deployment: nil) }
+ scope :with_project_route_and_deployment, -> { preload(:pages_deployment, project: [:namespace, :route]) }
+ end
+
+ # copy class with only required code
+ class PagesDeployment < ActiveRecord::Base
+ self.table_name = 'pages_deployments'
+ self.inheritance_column = :_type_disabled
+
+ include FileStoreMounter
+
+ attribute :file_store, :integer, default: -> { ::Pages::DeploymentUploader.default_store }
+
+ belongs_to :project, class_name: 'Project'
+
+ validates :file, presence: true
+ validates :file_store, presence: true, inclusion: { in: ObjectStorage::SUPPORTED_STORES }
+ validates :size, presence: true, numericality: { greater_than: 0, only_integer: true }
+ validates :file_count, presence: true, numericality: { greater_than_or_equal_to: 0, only_integer: true }
+ validates :file_sha256, presence: true
+
+ before_validation :set_size, if: :file_changed?
+
+ # Shall we copy all the uploader code migration as well?
+ mount_file_store_uploader ::Pages::DeploymentUploader
+
+ private
+
+ def set_size
+ self.size = file.size
+ end
+ end
- Also note, that
PagesDeployments
mounts::Pages::DeploymentUploader
, which in turn is based onGitlabUploader
,ObjectStorage::Concern
and requiresGitlab.config.pages
. - And we use services currently located under
app/services/pages/
I can try to move most of this under the lib/gitlab/background_migration/migrate_pages_to_zip_storage/
dir and reuse this code in the rake task, but I don't think it will make the migration more reliable.
Also the current approach was suggested in !54578 (comment 513143758)
Migration output
./bin/rails db:migrate:redo VERSION=20210302150310
== 20210302150310 ScheduleMigratePagesToZipStorage: reverting =================
== 20210302150310 ScheduleMigratePagesToZipStorage: reverted (0.0000s) ========
== 20210302150310 ScheduleMigratePagesToZipStorage: migrating =================
-- Scheduled 1 MigratePagesToZipStorage jobs with a maximum of 100 records per batch and an interval of 120 seconds.
The migration is expected to take at least 120 seconds. Expect all jobs to have completed after 2021-03-29 13:50:04 UTC."
== 20210302150310 ScheduleMigratePagesToZipStorage: migrated (0.0521s) ========
Database queries for the schedulling migration
SELECT "project_pages_metadata"."project_id"
FROM "project_pages_metadata"
WHERE "project_pages_metadata"."deployed" = TRUE
AND "project_pages_metadata"."pages_deployment_id" IS NULL
ORDER BY "project_pages_metadata"."project_id" ASC
LIMIT 1
https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/3119/commands/10272
SELECT "project_pages_metadata"."project_id"
FROM "project_pages_metadata"
WHERE "project_pages_metadata"."deployed" = TRUE
AND "project_pages_metadata"."pages_deployment_id" IS NULL
AND "project_pages_metadata"."project_id" >= 3707468
ORDER BY "project_pages_metadata"."project_id" ASC
LIMIT 1
OFFSET 100;
https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/3119/commands/10273
SELECT MIN(project_id),
MAX(project_id)
FROM "project_pages_metadata"
WHERE "project_pages_metadata"."deployed" = TRUE
AND "project_pages_metadata"."pages_deployment_id" IS NULL
AND "project_pages_metadata"."project_id" >= 3707468;
https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/3119/commands/10274
Database query for selecting batch(used in background migration)
SELECT "project_pages_metadata".*
FROM "project_pages_metadata"
WHERE "project_pages_metadata"."deployed" = TRUE
AND "project_pages_metadata"."pages_deployment_id" IS NULL
AND "project_pages_metadata"."project_id" BETWEEN 3707468 AND 24766231;
https://postgres.ai/console/gitlab/gitlab-production-tunnel/sessions/3518/commands/11644
Time estimation
We already migrated almost all of the gitlab.com(there are a few projects which failed to be deleted and marked as deployed), so in our own production this migration will be finished with only one iteration required, so it will take only a few minutes.
It took us about a week to run the migration by hand using 3 threads. So I'd say for instances with similar size this migration can take up to a months. (I don't think such instances exist though)
Rake migration updated output
vlad @ gdk1 ➜ gitlab git:(322001-poc-for-migrating-pages-to-zip-storage-in-the-background) ./bin/rake gitlab:pages:migrate_legacy_storage
I, [2021-04-15T14:45:26.167797 #1818942] INFO -- : Starting to migrate legacy pages storage to zip deployments
I, [2021-04-15T14:45:26.222246 #1818942] INFO -- : {:message=>"Pages legacy storage migration: Waiting for threads to finish..."}
E, [2021-04-15T14:45:26.859070 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2", :project_id=>3, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2", :duration=>0.11}
E, [2021-04-15T14:45:26.862268 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/demo-prepare", :project_id=>21, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/demo-prepare", :duration=>0.06}
E, [2021-04-15T14:45:26.862436 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/norbert/underscore", :project_id=>11, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/norbert/underscore", :duration=>0.12}
E, [2021-04-15T14:45:26.869745 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat", :project_id=>4, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat", :duration=>0.01}
E, [2021-04-15T14:45:26.872095 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test", :project_id=>23, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test", :duration=>0.01}
E, [2021-04-15T14:45:26.875534 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2", :project_id=>12, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2", :duration=>0.01}
E, [2021-04-15T14:45:26.882527 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore", :project_id=>5, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore", :duration=>0.01}
E, [2021-04-15T14:45:26.883321 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat", :project_id=>13, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat", :duration=>0.01}
E, [2021-04-15T14:45:26.886693 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2", :project_id=>24, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2", :duration=>0.01}
E, [2021-04-15T14:45:26.892480 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/flightjs/flight", :project_id=>6, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/flightjs/flight", :duration=>0.01}
E, [2021-04-15T14:45:26.895540 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat", :project_id=>14, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat", :duration=>0.01}
E, [2021-04-15T14:45:26.899355 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3", :project_id=>25, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3", :duration=>0.01}
E, [2021-04-15T14:45:26.905503 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js", :project_id=>7, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js", :duration=>0.01}
E, [2021-04-15T14:45:26.905667 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4", :project_id=>26, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4", :duration=>0.0}
E, [2021-04-15T14:45:26.908501 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat", :project_id=>15, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat", :duration=>0.01}
E, [2021-04-15T14:45:26.914845 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5", :project_id=>27, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5", :duration=>0.0}
E, [2021-04-15T14:45:26.918058 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate", :project_id=>8, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate", :duration=>0.01}
E, [2021-04-15T14:45:26.923015 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore", :project_id=>16, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore", :duration=>0.01}
E, [2021-04-15T14:45:26.924688 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/tony/lab-coat", :project_id=>9, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/tony/lab-coat", :duration=>0.0}
E, [2021-04-15T14:45:26.928938 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test", :project_id=>28, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test", :duration=>0.01}
E, [2021-04-15T14:45:26.935466 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore", :project_id=>17, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore", :duration=>0.01}
E, [2021-04-15T14:45:26.941866 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/alecia/wget2", :project_id=>10, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/alecia/wget2", :duration=>0.01}
E, [2021-04-15T14:45:26.942736 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks", :project_id=>30, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks", :duration=>0.01}
E, [2021-04-15T14:45:26.946356 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat", :project_id=>18, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat", :duration=>0.01}
E, [2021-04-15T14:45:26.953684 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell", :project_id=>2, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell", :duration=>0.01}
E, [2021-04-15T14:45:26.954405 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages-test", :project_id=>22, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/pages-test", :duration=>0.01}
E, [2021-04-15T14:45:26.955766 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-environments", :project_id=>19, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/prepare-environments", :duration=>0.01}
E, [2021-04-15T14:45:26.977514 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-demo", :project_id=>20, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/prepare-demo", :duration=>0.02}
I, [2021-04-15T14:45:26.977960 #1818942] INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>0, :errored=>28}
I, [2021-04-15T14:45:27.112577 #1818942] INFO -- : {:message=>"Pages legacy storage migration: project migrated: ", :project_id=>1, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test", :duration=>0.16}
I, [2021-04-15T14:45:27.112742 #1818942] INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>1, :errored=>28}
I, [2021-04-15T14:45:27.116354 #1818942] INFO -- : {:message=>"Pages legacy storage migration: project migrated: ", :project_id=>29, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/os-test", :duration=>0.16}
I, [2021-04-15T14:45:27.116664 #1818942] INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>2, :errored=>28}
E, [2021-04-15T14:45:27.146838 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks", :project_id=>31, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks", :duration=>0.0}
E, [2021-04-15T14:45:27.149848 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages", :project_id=>34, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/pages", :duration=>0.0}
E, [2021-04-15T14:45:27.152698 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/new-pages", :project_id=>35, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/new-pages", :duration=>0.0}
E, [2021-04-15T14:45:27.155623 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io", :project_id=>33, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io", :duration=>0.0}
E, [2021-04-15T14:45:27.158060 #1818942] ERROR -- : {:message=>"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test", :project_id=>32, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test", :duration=>0.0}
I, [2021-04-15T14:45:27.197230 #1818942] INFO -- : {:message=>"Pages legacy storage migration: project migrated: ", :project_id=>36, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/new-new-pages", :duration=>0.04}
I, [2021-04-15T14:45:27.197382 #1818942] INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>3, :errored=>33}
I, [2021-04-15T14:45:27.197970 #1818942] INFO -- : A total of 36 projects were processed.
I, [2021-04-15T14:45:27.198056 #1818942] INFO -- : - The 3 projects migrated successfully
I, [2021-04-15T14:45:27.198111 #1818942] INFO -- : - The 33 projects failed to be migrated
vlad @ gdk1 ➜ gitlab git:(322001-poc-for-migrating-pages-to-zip-storage-in-the-background) ✗ PAGES_MIGRATION_IGNORE_INVALID_ENTRIES=true PAGES_MIGRATION_MARK_PROJECTS_AS_NOT_DEPLOYED=true ./bin/rake gitlab:pages:migrate_legacy_storage
I, [2021-04-15T14:46:06.655333 #1819371] INFO -- : Starting to migrate legacy pages storage to zip deployments
I, [2021-04-15T14:46:06.713816 #1819371] INFO -- : {:message=>"Pages legacy storage migration: Waiting for threads to finish..."}
I, [2021-04-15T14:46:07.307180 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test? Marked project as not deployed", :project_id=>23, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test", :duration=>0.07}
I, [2021-04-15T14:46:07.308834 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/norbert/underscore? Marked project as not deployed", :project_id=>11, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/norbert/underscore", :duration=>0.09}
I, [2021-04-15T14:46:07.314805 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2? Marked project as not deployed", :project_id=>12, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2", :duration=>0.08}
I, [2021-04-15T14:46:07.324846 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat? Marked project as not deployed", :project_id=>13, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat", :duration=>0.01}
I, [2021-04-15T14:46:07.325667 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2? Marked project as not deployed", :project_id=>24, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2", :duration=>0.01}
I, [2021-04-15T14:46:07.327178 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2? Marked project as not deployed", :project_id=>3, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2", :duration=>0.02}
I, [2021-04-15T14:46:07.337607 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat? Marked project as not deployed", :project_id=>14, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat", :duration=>0.01}
I, [2021-04-15T14:46:07.342196 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3? Marked project as not deployed", :project_id=>25, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3", :duration=>0.01}
I, [2021-04-15T14:46:07.345880 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat? Marked project as not deployed", :project_id=>4, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat", :duration=>0.02}
I, [2021-04-15T14:46:07.353219 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat? Marked project as not deployed", :project_id=>15, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat", :duration=>0.01}
I, [2021-04-15T14:46:07.365904 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4? Marked project as not deployed", :project_id=>26, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4", :duration=>0.02}
I, [2021-04-15T14:46:07.371397 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore? Marked project as not deployed", :project_id=>5, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore", :duration=>0.02}
I, [2021-04-15T14:46:07.372261 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore? Marked project as not deployed", :project_id=>16, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore", :duration=>0.02}
I, [2021-04-15T14:46:07.390802 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5? Marked project as not deployed", :project_id=>27, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5", :duration=>0.02}
I, [2021-04-15T14:46:07.392840 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore? Marked project as not deployed", :project_id=>17, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore", :duration=>0.01}
I, [2021-04-15T14:46:07.399543 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/flightjs/flight? Marked project as not deployed", :project_id=>6, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/flightjs/flight", :duration=>0.02}
I, [2021-04-15T14:46:07.409148 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test? Marked project as not deployed", :project_id=>28, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test", :duration=>0.02}
I, [2021-04-15T14:46:07.410922 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat? Marked project as not deployed", :project_id=>18, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat", :duration=>0.02}
I, [2021-04-15T14:46:07.421209 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js? Marked project as not deployed", :project_id=>7, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js", :duration=>0.02}
I, [2021-04-15T14:46:07.429162 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/demo-prepare? Marked project as not deployed", :project_id=>21, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/demo-prepare", :duration=>0.02}
I, [2021-04-15T14:46:07.432642 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks? Marked project as not deployed", :project_id=>30, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks", :duration=>0.02}
I, [2021-04-15T14:46:07.448571 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate? Marked project as not deployed", :project_id=>8, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate", :duration=>0.02}
I, [2021-04-15T14:46:07.455081 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-environments? Marked project as not deployed", :project_id=>19, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/prepare-environments", :duration=>0.02}
I, [2021-04-15T14:46:07.460717 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages-test? Marked project as not deployed", :project_id=>22, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/pages-test", :duration=>0.02}
I, [2021-04-15T14:46:07.475750 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/tony/lab-coat? Marked project as not deployed", :project_id=>9, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/tony/lab-coat", :duration=>0.02}
I, [2021-04-15T14:46:07.482210 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-demo? Marked project as not deployed", :project_id=>20, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/prepare-demo", :duration=>0.02}
I, [2021-04-15T14:46:07.484969 #1819371] INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>26, :errored=>0}
I, [2021-04-15T14:46:07.485870 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks? Marked project as not deployed", :project_id=>31, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks", :duration=>0.02}
I, [2021-04-15T14:46:07.496882 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/alecia/wget2? Marked project as not deployed", :project_id=>10, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/alecia/wget2", :duration=>0.02}
I, [2021-04-15T14:46:07.513568 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test? Marked project as not deployed", :project_id=>32, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test", :duration=>0.01}
I, [2021-04-15T14:46:07.513888 #1819371] INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>29, :errored=>0}
I, [2021-04-15T14:46:07.523019 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell? Marked project as not deployed", :project_id=>2, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell", :duration=>0.02}
I, [2021-04-15T14:46:07.523416 #1819371] INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>30, :errored=>0}
I, [2021-04-15T14:46:07.579295 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages? Marked project as not deployed", :project_id=>34, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/pages", :duration=>0.01}
I, [2021-04-15T14:46:07.586227 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/new-pages? Marked project as not deployed", :project_id=>35, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/root/new-pages", :duration=>0.01}
I, [2021-04-15T14:46:07.594834 #1819371] INFO -- : {:message=>"Pages legacy storage migration: project migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io? Marked project as not deployed", :project_id=>33, :pages_path=>"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io", :duration=>0.01}
I, [2021-04-15T14:46:07.595163 #1819371] INFO -- : {:message=>"Pages legacy storage migration: batch processed", :migrated=>33, :errored=>0}
I, [2021-04-15T14:46:07.595741 #1819371] INFO -- : A total of 33 projects were processed.
I, [2021-04-15T14:46:07.595974 #1819371] INFO -- : - The 33 projects migrated successfully
I, [2021-04-15T14:46:07.596187 #1819371] INFO -- : - The 0 projects failed to be migrated
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Background migration logs
(I changed batch size to 20, and time to 2 minutes to speed up tests locally)
vlad @ gdk1 ➜ gitlab git:(322001-poc-for-migrating-pages-to-zip-storage-in-the-background) ✗ tail -F log/application_json.log | grep "legacy"
{"severity":"ERROR","time":"2021-04-15T13:03:11.071Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/norbert/underscore","project_id":11,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/norbert/underscore","duration":0.02}
{"severity":"ERROR","time":"2021-04-15T13:03:11.074Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2","project_id":12,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/marquita_kshlerin/wget2","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.077Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2","project_id":3,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gnuwget/wget2","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.080Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat","project_id":13,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/neda.tromp/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.085Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat","project_id":14,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/breana.schuster/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.090Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat","project_id":4,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/Commit451/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.094Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat","project_id":15,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/ronda/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.097Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore","project_id":5,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/jashkenas/underscore","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.100Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore","project_id":16,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gerald_lindgren/underscore","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.103Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/flightjs/flight","project_id":6,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/flightjs/flight","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.105Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore","project_id":17,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/jama_grimes/underscore","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.108Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat","project_id":18,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/serita.crona/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.111Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js","project_id":7,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/twitter/typeahead-js","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.114Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate","project_id":8,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/h5bp/html5-boilerplate","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.117Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-environments","project_id":19,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/prepare-environments","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.120Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/tony/lab-coat","project_id":9,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/tony/lab-coat","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.122Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/prepare-demo","project_id":20,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/prepare-demo","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.126Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/alecia/wget2","project_id":10,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/alecia/wget2","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:03:11.128Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell","project_id":2,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-shell","duration":0.0}
{"severity":"INFO","time":"2021-04-15T13:03:11.246Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: project migrated: ","project_id":1,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test","duration":0.12}
{"severity":"INFO","time":"2021-04-15T13:03:11.246Z","correlation_id":"1c65f633435e469053be3b1ab182eed7","message":"Pages legacy storage migration: batch processed","migrated":1,"errored":19}
{"severity":"ERROR","time":"2021-04-15T13:05:10.402Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test","project_id":23,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.408Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2","project_id":24,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/getsby-test-2","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.413Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3","project_id":25,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-3","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.418Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4","project_id":26,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-4","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.421Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5","project_id":27,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/gatsby-test-5","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.425Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test","project_id":28,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/objectstorage.pages.test","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.428Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks","project_id":30,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/test-deployment-webhooks","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.431Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/demo-prepare","project_id":21,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/demo-prepare","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.433Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages-test","project_id":22,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/pages-test","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.435Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks","project_id":31,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/webhooks-group/test-group-webhooks","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.437Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test","project_id":32,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/pages-test","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.439Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/pages","project_id":34,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/pages","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.441Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/root/new-pages","project_id":35,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/new-pages","duration":0.0}
{"severity":"ERROR","time":"2021-04-15T13:05:10.443Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project failed to be migrated: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io","project_id":33,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-org.10.126.0.2.nip.io","duration":0.0}
{"severity":"INFO","time":"2021-04-15T13:05:10.489Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project migrated: ","project_id":29,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/os-test","duration":0.04}
{"severity":"INFO","time":"2021-04-15T13:05:10.529Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: project migrated: ","project_id":36,"pages_path":"/home/vlad/gdk/gitlab/shared/pages/root/new-new-pages","duration":0.04}
{"severity":"INFO","time":"2021-04-15T13:05:10.530Z","correlation_id":"d1a19333ea805745e3cd74f41af76f07","message":"Pages legacy storage migration: batch processed","migrated":2,"errored":14}
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Related to #322001 (closed)
Merge request reports
- version 52b8e3b435
- version 512271ab60
- version 50b3362d03
- version 49593177d0
- version 487f393143
- version 47f4e89033
- version 46f48a8cb6
- version 45efba4c1e
- version 446944a0b6
- version 437b3f4d52
- version 423a3c7af4
- version 4151052dbe
- version 40d05be98c
- version 39d05be98c
- version 385b2a6b31
- version 373dbef7b0
- version 360c00f5b4
- version 35c5b07756
- version 348b68428b
- version 332ff10c92
- version 32ac3a233f
- version 31ac3a233f
- version 30ac3a233f
- version 29ac3a233f
- version 28d0e35301
- version 2700b98074
- version 2600b98074
- version 250d46a1c8
- version 241df24fe0
- version 2325102bab
- version 22c33a2724
- version 219b34dab8
- version 203b3cbbc5
- version 19888ba9a1
- version 18feacae02
- version 17ad957154
- version 16b247dbc0
- version 15372dfda9
- version 149dd54a20
- version 13bde1b3d2
- version 125d0bc221
- version 115d0bc221
- version 10c8fadccd
- version 9aa9a32dc
- version 8e6af34c6
- version 7b89ab0a9
- version 6ca94dcb1
- version 585b11836
- version 4cd29479a
- version 3310f2781
- version 286cf9d4f
- version 16e1ece7f
- master (base)
- latest version08cec96c4 commits,
- version 52b8e3b4353 commits,
- version 512271ab601 commit,
- version 50b3362d031 commit,
- version 49593177d01 commit,
- version 487f3931431 commit,
- version 47f4e890331 commit,
- version 46f48a8cb61 commit,
- version 45efba4c1e1 commit,
- version 446944a0b62 commits,
- version 437b3f4d521 commit,
- version 423a3c7af41 commit,
- version 4151052dbe1 commit,
- version 40d05be98c1 commit,
- version 39d05be98c1 commit,
- version 385b2a6b311 commit,
- version 373dbef7b01 commit,
- version 360c00f5b44 commits,
- version 35c5b077563 commits,
- version 348b68428b2 commits,
- version 332ff10c921 commit,
- version 32ac3a233f2 commits,
- version 31ac3a233f2 commits,
- version 30ac3a233f1 commit,
- version 29ac3a233f2 commits,
- version 28d0e353011 commit,
- version 2700b980741 commit,
- version 2600b980741 commit,
- version 250d46a1c81 commit,
- version 241df24fe01 commit,
- version 2325102bab1 commit,
- version 22c33a27242 commits,
- version 219b34dab82 commits,
- version 203b3cbbc51 commit,
- version 19888ba9a11 commit,
- version 18feacae022 commits,
- version 17ad9571542 commits,
- version 16b247dbc01 commit,
- version 15372dfda91 commit,
- version 149dd54a201 commit,
- version 13bde1b3d21 commit,
- version 125d0bc2211 commit,
- version 115d0bc2211 commit,
- version 10c8fadccd1 commit,
- version 9aa9a32dc2 commits,
- version 8e6af34c62 commits,
- version 7b89ab0a91 commit,
- version 6ca94dcb11 commit,
- version 585b118361 commit,
- version 4cd29479a1 commit,
- version 3310f27811 commit,
- version 286cf9d4f1 commit,
- version 16e1ece7f1 commit,
- Side-by-side
- Inline