Skip to content

Set `parent_id` to nil on namespaces whose parent is now non existent

What does this MR do and why?

Based on the #419492, we want to resolve the 500 error problem.

I've dug into the trace, and identify that issue is related to ee/lib/gitlab/auth/group_saml/sso_enforcer.rb, where we're calling root_saml_provider without checking if root_ancestor is present.

Going back to the changes, I found Refactor SSO enforcement related code (!120105 - merged), that removed the following code:

-- return false unless group.root_ancestor

and the issue was opened slightly after this change.

After giving a second view, we came to the conclusion that we're having some data integrity concerns, therefore we're introducing batched_background_migrations to nullify parent_id from namespaces table.

Currently, we're having 34 entries, which should be nullified.

Deployment steps per add foreign key to existing column

  1. GitLab version N.M: Add a NOT VALID foreign key constraint to the column to ensure GitLab doesn’t create inconsistent records. Add not valid foreign key to namespaces.parent_id (!153402 - merged)
  2. GitLab version N.M: Add a data migration, to fix or clean up existing records. Set `parent_id` to nil on namespaces whose pare... (!153101)
  3. GitLab version N.M+1: Validate the whole table by making the foreign key VALID. TBD

MR acceptance checklist

DB query. - 34 matches

Query Execution
SELECT n1.id FROM namespaces n1 LEFT JOIN namespaces n2 ON n1.parent_id = n2.id WHERE n1.parent_id IS NOT NULL AND n2.id IS NULL; https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/28285/commands/88277

Up migrations

marjanovic@Bojans-MacBook-Pro gitlab % rails db:migrate
main: == [advisory_lock_connection] object_id: 125420, pg_backend_pid: 68009
main: == 20240516085240 QueueNullifyParentIdForNamespaces: migrating ================
main: == 20240516085240 QueueNullifyParentIdForNamespaces: migrated (0.1018s) =======

main: == [advisory_lock_connection] object_id: 125420, pg_backend_pid: 68009
ci: == [advisory_lock_connection] object_id: 126080, pg_backend_pid: 68011
ci: == 20240516085240 QueueNullifyParentIdForNamespaces: migrating ================
ci: -- The migration is skipped since it modifies the schemas: [:gitlab_main].
ci: -- This database can only apply migrations in one of the following schemas: [:gitlab_ci, :gitlab_internal, :gitlab_shared].
ci: == 20240516085240 QueueNullifyParentIdForNamespaces: migrated (0.0208s) =======

ci: == [advisory_lock_connection] object_id: 126080, pg_backend_pid: 68011

Down migrations

marjanovic@Bojans-MacBook-Pro gitlab % bundle exec rails db:rollback:main STEP=1    
main: == [advisory_lock_connection] object_id: 125020, pg_backend_pid: 94014
main: == 20240516085240 QueueNullifyParentIdForNamespaces: reverting ================
main: == 20240516085240 QueueNullifyParentIdForNamespaces: reverted (0.0552s) =======

main: == [advisory_lock_connection] object_id: 125020, pg_backend_pid: 94014
marjanovic@Bojans-MacBook-Pro gitlab % bundle exec rails db:rollback:ci STEP=1
ci: == [advisory_lock_connection] object_id: 125020, pg_backend_pid: 94619
ci: == 20240516085240 QueueNullifyParentIdForNamespaces: reverting ================
ci: -- The migration is skipped since it modifies the schemas: [:gitlab_main].
ci: -- This database can only apply migrations in one of the following schemas: [:gitlab_ci, :gitlab_internal, :gitlab_shared].
ci: == 20240516085240 QueueNullifyParentIdForNamespaces: reverted (0.0111s) =======

ci: == [advisory_lock_connection] object_id: 125020, pg_backend_pid: 94619

How to set up and validate locally

  1. Checkout the branches
  2. Check there are some inconsistent records in database (non-existing parent_id in namespaces table)
  3. Run the migrations rails db:migrate
  4. Verify that all records have been fixed 🎉

Related to #419492

Edited by Bojan Marjanovic

Merge request reports