Skip to content

Background migration to copy projects.container_registry_enabled to project_features.container_registry_access_level [RUN ALL RSPEC]

Reuben Pereira requested to merge 18792-background-migration into master

What does this MR do?

Adds a background migration to copy projects.container_registry_enabled to project_features.container_registry_access_level:

  1. If projects.container_registry_enabled is true, sets project_features.container_registry_access_level to 20.
  2. If projects.container_registry_enabled is false, sets project_features.container_registry_access_level to 0.
  3. If projects.container_registry_enabled is null, sets project_features.container_registry_access_level to 0.

This MR also:

  1. Adds a before_update callback to Project model, which copies any changes to the projects.container_registry_enabled column to the project_features.container_registry_access_level column.
  2. Adds a before_create callback to the ProjectFeature model which sets the value of the container_registry_access_level based on the value of projects.container_registry_enabled.

This is the 2nd in a series of MRs. Please see #18792 (comment 519028655) for more details:

  1. !54831 (merged)
  2. This MR.
  3. !55071 (merged)
  4. !55077 (closed)
  5. !55079 (merged)

Background migration estimates

projects table has 18,189,661 rows

All rows to be migrated.

50,000 rows per sidekiq job

18,189,661 / 50,000 = 364 jobs

1,000 rows per batch in job

50,000 / 1,000 = 50 batches per job

18,189,661 / 1,000 = 18,190 total batches

Estimated times per batch:

1466ms for update statement with 1000 items (from https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/2565/commands/7900)

Execution time per sidekiq job:
1466 * 50 = 73.3 seconds

Sidekiq jobs are scheduled 2 minutes apart.

364 * 120 = 728 minutes = ~12.1 hours

Migration output

== 20210226120851 MoveContainerRegistryEnabledToProjectFeatures: migrating ====
-- Scheduled 1 MoveContainerRegistryEnabledToProjectFeature jobs with a maximum of 50000 records per batch and an interval of 120 seconds.

The migration is expected to take at least 120 seconds. Expect all jobs to have completed after 2021-03-08 19:24:33 UTC."
== 20210226120851 MoveContainerRegistryEnabledToProjectFeatures: migrated (0.1167s)

Revert migration output:

== 20210226120851 MoveContainerRegistryEnabledToProjectFeatures: reverting ====
== 20210226120851 MoveContainerRegistryEnabledToProjectFeatures: reverted (0.0000s)

Postgres.ai link for background migration update query

https://console.postgres.ai/gitlab/gitlab-production-tunnel/sessions/2565/commands/7900

Related to #18792 (closed)

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Reuben Pereira

Merge request reports