Skip to content

Resolve "Update existing public projects in private groups to private projects"

Gosia Ksionek requested to merge 22406-update-existing-public-projects-in-private-groups-to-private-projects-new into master

Adds migration that updates all projects, that have a higher visibility level than the group they are in. After some discussion it was advised to move it to the background migration.

First it was developed here but due to the long discussions and a lot of time that passed since I started it, I decided to create fresh branch.

Screenshots (strongly suggested)

Database-lab results: PRIVATE LEVEL query:

explain SELECT projects.id FROM projects INNER JOIN namespaces ON namespaces.id = projects.namespace_id WHERE namespaces.type = 'Group' AND namespaces.visibility_level = 0 AND (projects.visibility_level > 0) AND projects.id >= 1

time: (cold cash)

Time: 4.339 min
  - planning: 0.971 ms
  - execution: 4.339 min
    - I/O read: 12.496 min
    - I/O write: 0.081 ms

Shared buffers:
  - hits: 7451341 (~56.80 GiB) from the buffer pool
  - reads: 4427447 (~33.80 GiB) from the OS file cache, including disk I/O
  - dirtied: 24769 (~193.50 MiB)
  - writes: 1 (~8.00 KiB)

plan: https://explain.depesz.com/s/zaGz

Internal LEVEL query:

explain SELECT projects.id FROM projects INNER JOIN namespaces ON namespaces.id = projects.namespace_id WHERE namespaces.type = 'Group' AND namespaces.visibility_level = 10 AND (projects.visibility_level > 10) AND projects.id >= 1

time: (cold cash)

Time: 1.564 min
  - planning: 1.282 ms
  - execution: 1.564 min
    - I/O read: 4.513 min
    - I/O write: N/A

Shared buffers:
  - hits: 922788 (~7.00 GiB) from the buffer pool
  - reads: 308557 (~2.40 GiB) from the OS file cache, including disk I/O
  - dirtied: 13 (~104.00 KiB)
  - writes: 0

plan: https://explain.depesz.com/s/L59y

With smaller batch: PRIVATE LEVEL query:

explain SELECT projects.id FROM projects INNER JOIN namespaces ON namespaces.id = projects.namespace_id WHERE namespaces.type = 'Group' AND namespaces.visibility_level = 0 AND (projects.visibility_level > 0) AND projects.id >= 1 AND projects.id < 1000

time: (cold cash)

Time: 507.004 ms
  - planning: 0.980 ms
  - execution: 506.024 ms
    - I/O read: 501.347 ms
    - I/O write: N/A

Shared buffers:
  - hits: 60 (~480.00 KiB) from the buffer pool
  - reads: 413 (~3.20 MiB) from the OS file cache, including disk I/O
  - dirtied: 2 (~16.00 KiB)
  - writes: 0

plan: https://explain.depesz.com/s/q26M

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team

Related to #22406 (closed)

Edited by Gosia Ksionek

Merge request reports