Sync organization_id to CI mirror tables
What does this MR do and why?
This MR extends the CI Mirroring so it will also sync organization_id to projects and namespaces mirror tables.
This is needed for organization isolation. The logic is relying on joining namespaces or projects table. Since these tables are not present on ci databases, we need to sync the organization_id to both mirror tables.
This MR is part 2 of a series:
- !253978 (merged) - add the
organization_idcolumns - This MR - sync
organization_idon changes - !254209 (merged) - backfill
organization_idfor existing rows - !254210 - use the mirror tables in the data isolation logic
References
- Related to #627588
Query plans
INSERT INTO "ci_namespace_mirrors" ("namespace_id","traversal_ids","organization_id")
VALUES (10, '{1,2,10}', 1000) ON CONFLICT ("namespace_id")
DO UPDATE SET "traversal_ids"=excluded."traversal_ids","organization_id"=excluded."organization_id"
RETURNING "id"https://console.postgres.ai/gitlab/projects/gitlab-production-ci/sessions/56684/commands/161038
Time: 20.607 ms
- planning: 0.360 ms
- execution: 20.247 ms
- I/O read: 19.232 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 78 (~624.00 KiB) from the buffer pool
- reads: 26 (~208.00 KiB) from the OS file cache, including disk I/O
- dirtied: 9 (~72.00 KiB)
- writes: 0INSERT INTO "ci_project_mirrors" ("project_id","namespace_id","organization_id")
VALUES (17, 10, 1000) ON CONFLICT ("project_id")
DO UPDATE SET "namespace_id"=excluded."namespace_id","organization_id"=excluded."organization_id"
RETURNING "id"https://postgres.ai/console/gitlab/gitlab-production-ci/sessions/56684/commands/161041
Time: 38.689 ms
- planning: 0.354 ms
- execution: 38.335 ms
- I/O read: 37.209 ms
- I/O write: 0.000 ms
Shared buffers:
- hits: 39 (~312.00 KiB) from the buffer pool
- reads: 19 (~152.00 KiB) from the OS file cache, including disk I/O
- dirtied: 7 (~56.00 KiB)
- writes: 0How to set up and validate locally
Open two terminals:
- One with a Rails console:
gdk rails c - One with a PostgreSQL console:
gdk psql -d gitlabhq_development_ci
In the PostgreSQL console:
gitlabhq_development_ci=# select * from ci_project_mirrors;select * from ci_namespace_mirrors;Now we are going to move the 'twitter' group to a new organization. In the Rails console:
user = User.find_by_username('root')
my_org = Organizations::Organization.find_or_create_by!(path: 'my-org') { |org| org.name = 'My Org' }
group = Group.find_by_path('twitter')
Organizations::Transfer::GroupsService.new(group: group, new_organization: my_org, current_user: user).execute
::Projects::ProcessSyncEventsWorker.perform_sync
::Namespaces::ProcessSyncEventsWorker.perform_syncAnd now in the PostgreSQL console, both mirror tables should show the new organization_id for the moved group and its projects.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.