Sync organization_id to CI mirror tables

What does this MR do and why?

This MR extends the CI Mirroring so it will also sync organization_id to projects and namespaces mirror tables.

This is needed for organization isolation. The logic is relying on joining namespaces or projects table. Since these tables are not present on ci databases, we need to sync the organization_id to both mirror tables.

This MR is part 2 of a series:

  1. !253978 (merged) - add the organization_id columns
  2. This MR - sync organization_id on changes
  3. !254209 (merged) - backfill organization_id for existing rows
  4. !254210 - use the mirror tables in the data isolation logic

References

Query plans

INSERT INTO "ci_namespace_mirrors" ("namespace_id","traversal_ids","organization_id")
VALUES (10, '{1,2,10}', 1000) ON CONFLICT ("namespace_id")
DO UPDATE SET "traversal_ids"=excluded."traversal_ids","organization_id"=excluded."organization_id"
RETURNING "id"

https://console.postgres.ai/gitlab/projects/gitlab-production-ci/sessions/56684/commands/161038

Time: 20.607 ms
  - planning: 0.360 ms
  - execution: 20.247 ms
    - I/O read: 19.232 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 78 (~624.00 KiB) from the buffer pool
  - reads: 26 (~208.00 KiB) from the OS file cache, including disk I/O
  - dirtied: 9 (~72.00 KiB)
  - writes: 0
INSERT INTO "ci_project_mirrors" ("project_id","namespace_id","organization_id")
VALUES (17, 10, 1000) ON CONFLICT ("project_id")
DO UPDATE SET "namespace_id"=excluded."namespace_id","organization_id"=excluded."organization_id"
RETURNING "id"

https://postgres.ai/console/gitlab/gitlab-production-ci/sessions/56684/commands/161041

Time: 38.689 ms
  - planning: 0.354 ms
  - execution: 38.335 ms
    - I/O read: 37.209 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 39 (~312.00 KiB) from the buffer pool
  - reads: 19 (~152.00 KiB) from the OS file cache, including disk I/O
  - dirtied: 7 (~56.00 KiB)
  - writes: 0

How to set up and validate locally

Open two terminals:

  • One with a Rails console: gdk rails c
  • One with a PostgreSQL console: gdk psql -d gitlabhq_development_ci

In the PostgreSQL console:

gitlabhq_development_ci=# select * from ci_project_mirrors;select * from ci_namespace_mirrors;

Now we are going to move the 'twitter' group to a new organization. In the Rails console:

user = User.find_by_username('root')
my_org = Organizations::Organization.find_or_create_by!(path: 'my-org') { |org| org.name = 'My Org' }
group = Group.find_by_path('twitter')
Organizations::Transfer::GroupsService.new(group: group, new_organization: my_org, current_user: user).execute
::Projects::ProcessSyncEventsWorker.perform_sync
::Namespaces::ProcessSyncEventsWorker.perform_sync

And now in the PostgreSQL console, both mirror tables should show the new organization_id for the moved group and its projects.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Rutger Wessels

Merge request reports

Loading
Loading