Migrate admin_bot to per-organization internal user
What does this MR do and why?
Migrate admin_bot to per-organization internal user
With the change to make internal bot users per-organization, we must begin migrating all calls to Users::Internal to include a reference organization. This change updates all calls to admin_bot to follow this process.
Changelog: changed
References
- Original MR this is split out from: !186023 (closed) and #558806 (closed)
- Feature issue: #442780 (closed)
- Part 1: Allow specifying an Organization for internal users: !186690 (merged)
- Part 2: Add OrganizationUserAlias table: !189052 (merged)
- Part 3: Implement OrganizationUserDetails for internal bot users: !190607 (merged)
Screenshots or screen recordings
| Before | After |
|---|---|
Database Query Changes
Users::Internal.admin_bot
Database query
SELECT "users"."id",
"users"."email",
"users"."encrypted_password",
"users"."reset_password_token",
"users"."reset_password_sent_at",
"users"."remember_created_at",
"users"."sign_in_count",
"users"."current_sign_in_at",
"users"."last_sign_in_at",
"users"."current_sign_in_ip",
"users"."last_sign_in_ip",
"users"."created_at",
"users"."updated_at",
"users"."name",
"users"."admin",
"users"."projects_limit",
"users"."failed_attempts",
"users"."locked_at",
"users"."username",
"users"."can_create_group",
"users"."can_create_team",
"users"."state",
"users"."color_scheme_id",
"users"."password_expires_at",
"users"."created_by_id",
"users"."last_credential_check_at",
"users"."avatar",
"users"."confirmation_token",
"users"."confirmed_at",
"users"."confirmation_sent_at",
"users"."unconfirmed_email",
"users"."hide_no_ssh_key",
"users"."admin_email_unsubscribed_at",
"users"."notification_email",
"users"."hide_no_password",
"users"."password_automatically_set",
"users"."encrypted_otp_secret",
"users"."encrypted_otp_secret_iv",
"users"."encrypted_otp_secret_salt",
"users"."otp_required_for_login",
"users"."otp_backup_codes",
"users"."public_email",
"users"."dashboard",
"users"."project_view",
"users"."consumed_timestep",
"users"."layout",
"users"."hide_project_limit",
"users"."note",
"users"."unlock_token",
"users"."otp_grace_period_started_at",
"users"."external",
"users"."incoming_email_token",
"users"."auditor",
"users"."require_two_factor_authentication_from_group",
"users"."two_factor_grace_period",
"users"."last_activity_on",
"users"."notified_of_own_activity",
"users"."preferred_language",
"users"."theme_id",
"users"."accepted_term_id",
"users"."feed_token",
"users"."private_profile",
"users"."roadmap_layout",
"users"."include_private_contributions",
"users"."commit_email",
"users"."group_view",
"users"."managing_group_id",
"users"."first_name",
"users"."last_name",
"users"."static_object_token",
"users"."user_type",
"users"."static_object_token_encrypted",
"users"."otp_secret_expires_at",
"users"."onboarding_in_progress",
"users"."color_mode_id",
"users"."composite_identity_enforced",
"users"."organization_id"
FROM "users"
WHERE "users"."user_type" = 11
ORDER BY "users"."id" ASC LIMIT 1https://postgres.ai/console/gitlab/gitlab-production-main/sessions/41764/commands/128167
Users::Internal.for_organization(my_organization).admin_bot
Database query
SELECT "users"."id",
"users"."email",
"users"."encrypted_password",
"users"."reset_password_token",
"users"."reset_password_sent_at",
"users"."remember_created_at",
"users"."sign_in_count",
"users"."current_sign_in_at",
"users"."last_sign_in_at",
"users"."current_sign_in_ip",
"users"."last_sign_in_ip",
"users"."created_at",
"users"."updated_at",
"users"."name",
"users"."admin",
"users"."projects_limit",
"users"."failed_attempts",
"users"."locked_at",
"users"."username",
"users"."can_create_group",
"users"."can_create_team",
"users"."state",
"users"."color_scheme_id",
"users"."password_expires_at",
"users"."created_by_id",
"users"."last_credential_check_at",
"users"."avatar",
"users"."confirmation_token",
"users"."confirmed_at",
"users"."confirmation_sent_at",
"users"."unconfirmed_email",
"users"."hide_no_ssh_key",
"users"."admin_email_unsubscribed_at",
"users"."notification_email",
"users"."hide_no_password",
"users"."password_automatically_set",
"users"."encrypted_otp_secret",
"users"."encrypted_otp_secret_iv",
"users"."encrypted_otp_secret_salt",
"users"."otp_required_for_login",
"users"."otp_backup_codes",
"users"."public_email",
"users"."dashboard",
"users"."project_view",
"users"."consumed_timestep",
"users"."layout",
"users"."hide_project_limit",
"users"."note",
"users"."unlock_token",
"users"."otp_grace_period_started_at",
"users"."external",
"users"."incoming_email_token",
"users"."auditor",
"users"."require_two_factor_authentication_from_group",
"users"."two_factor_grace_period",
"users"."last_activity_on",
"users"."notified_of_own_activity",
"users"."preferred_language",
"users"."theme_id",
"users"."accepted_term_id",
"users"."feed_token",
"users"."private_profile",
"users"."roadmap_layout",
"users"."include_private_contributions",
"users"."commit_email",
"users"."group_view",
"users"."managing_group_id",
"users"."first_name",
"users"."last_name",
"users"."static_object_token",
"users"."user_type",
"users"."static_object_token_encrypted",
"users"."otp_secret_expires_at",
"users"."onboarding_in_progress",
"users"."color_mode_id",
"users"."composite_identity_enforced",
"users"."organization_id"
FROM "users"
INNER JOIN "organization_users" ON "organization_users"."user_id" = "users"."id"
WHERE "users"."user_type" = 11
AND "organization_users"."organization_id" = 1
ORDER BY "users"."id" ASC LIMIT 1https://postgres.ai/console/gitlab/gitlab-production-main/sessions/41764/commands/128169
pipl_deletable Scope Changes
This scope is only used by DeletePiplUsersWorker , so I tried to reproduce the batch-query behavior exhibited here. I created two ComplianceManagement::PiplUsers locally and got the following queries.
Current behavior:
Get batch
SELECT COUNT(DISTINCT "pipl_users"."user_id")
FROM "pipl_users"
INNER JOIN "users" ON "users"."id" = "pipl_users"."user_id"
LEFT OUTER JOIN "ghost_user_migrations" ON "ghost_user_migrations"."user_id" = "users"."id"
WHERE "ghost_user_migrations"."id" IS NULL
AND "pipl_users"."state" != 1
AND "pipl_users"."initial_email_sent_at" <= '2025-03-27 23:59:59.999999'
AND ("users"."state" IN ('blocked',
'ldap_blocked'))https://postgres.ai/console/gitlab/gitlab-production-main/sessions/41858/commands/128426
Get User
SELECT "users"."id",
"users"."email",
"users"."encrypted_password",
"users"."reset_password_token",
"users"."reset_password_sent_at",
"users"."remember_created_at",
"users"."sign_in_count",
"users"."current_sign_in_at",
"users"."last_sign_in_at",
"users"."current_sign_in_ip",
"users"."last_sign_in_ip",
"users"."created_at",
"users"."updated_at",
"users"."name",
"users"."admin",
"users"."projects_limit",
"users"."failed_attempts",
"users"."locked_at",
"users"."username",
"users"."can_create_group",
"users"."can_create_team",
"users"."state",
"users"."color_scheme_id",
"users"."password_expires_at",
"users"."created_by_id",
"users"."last_credential_check_at",
"users"."avatar",
"users"."confirmation_token",
"users"."confirmed_at",
"users"."confirmation_sent_at",
"users"."unconfirmed_email",
"users"."hide_no_ssh_key",
"users"."admin_email_unsubscribed_at",
"users"."notification_email",
"users"."hide_no_password",
"users"."password_automatically_set",
"users"."encrypted_otp_secret",
"users"."encrypted_otp_secret_iv",
"users"."encrypted_otp_secret_salt",
"users"."otp_required_for_login",
"users"."otp_backup_codes",
"users"."public_email",
"users"."dashboard",
"users"."project_view",
"users"."consumed_timestep",
"users"."layout",
"users"."hide_project_limit",
"users"."note",
"users"."unlock_token",
"users"."otp_grace_period_started_at",
"users"."external",
"users"."incoming_email_token",
"users"."auditor",
"users"."require_two_factor_authentication_from_group",
"users"."two_factor_grace_period",
"users"."last_activity_on",
"users"."notified_of_own_activity",
"users"."preferred_language",
"users"."theme_id",
"users"."accepted_term_id",
"users"."feed_token",
"users"."private_profile",
"users"."roadmap_layout",
"users"."include_private_contributions",
"users"."commit_email",
"users"."group_view",
"users"."managing_group_id",
"users"."first_name",
"users"."last_name",
"users"."static_object_token",
"users"."user_type",
"users"."static_object_token_encrypted",
"users"."otp_secret_expires_at",
"users"."onboarding_in_progress",
"users"."color_mode_id",
"users"."composite_identity_enforced",
"users"."organization_id"
FROM "users"
WHERE "users"."user_type" = 11
ORDER BY "users"."id" ASC LIMIT 1https://postgres.ai/console/gitlab/gitlab-production-main/sessions/41858/commands/128427
It looks like the "Get User" query here may be an already-existing N+1 query.
Behavior with new includes option:
Get batch sizes
SELECT "pipl_users"."user_id"
FROM "pipl_users"
INNER JOIN "users" ON "users"."id" = "pipl_users"."user_id"
LEFT OUTER JOIN "ghost_user_migrations" ON "ghost_user_migrations"."user_id" = "users"."id"
WHERE "ghost_user_migrations"."id" IS NULL
AND "pipl_users"."state" != 1
AND "pipl_users"."initial_email_sent_at" <= '2025-03-27 23:59:59.999999'
AND ("users"."state" IN ('blocked',
'ldap_blocked'))
ORDER BY "pipl_users"."user_id" ASC LIMIT 1https://postgres.ai/console/gitlab/gitlab-production-main/sessions/41858/commands/128428
Get current batch of PiplUser
SELECT "pipl_users"."user_id"
FROM "pipl_users"
INNER JOIN "users" ON "users"."id" = "pipl_users"."user_id"
LEFT OUTER JOIN "ghost_user_migrations" ON "ghost_user_migrations"."user_id" = "users"."id"
WHERE "ghost_user_migrations"."id" IS NULL
AND "pipl_users"."state" != 1
AND "pipl_users"."initial_email_sent_at" <= '2025-03-27 23:59:59.999999'
AND ("users"."state" IN ('blocked',
'ldap_blocked'))
AND "pipl_users"."user_id" >= 11
ORDER BY "pipl_users"."user_id" ASC LIMIT 1
OFFSET 1000https://postgres.ai/console/gitlab/gitlab-production-main/sessions/41858/commands/128429
Preload organizations
SELECT "organizations".* FROM "organizations" WHERE "organizations"."id" = 1https://postgres.ai/console/gitlab/gitlab-production-main/sessions/41858/commands/128430
Preload users
SELECT "users"."id",
"users"."email",
"users"."encrypted_password",
"users"."reset_password_token",
"users"."reset_password_sent_at",
"users"."remember_created_at",
"users"."sign_in_count",
"users"."current_sign_in_at",
"users"."last_sign_in_at",
"users"."current_sign_in_ip",
"users"."last_sign_in_ip",
"users"."created_at",
"users"."updated_at",
"users"."name",
"users"."admin",
"users"."projects_limit",
"users"."failed_attempts",
"users"."locked_at",
"users"."username",
"users"."can_create_group",
"users"."can_create_team",
"users"."state",
"users"."color_scheme_id",
"users"."password_expires_at",
"users"."created_by_id",
"users"."last_credential_check_at",
"users"."avatar",
"users"."confirmation_token",
"users"."confirmed_at",
"users"."confirmation_sent_at",
"users"."unconfirmed_email",
"users"."hide_no_ssh_key",
"users"."admin_email_unsubscribed_at",
"users"."notification_email",
"users"."hide_no_password",
"users"."password_automatically_set",
"users"."encrypted_otp_secret",
"users"."encrypted_otp_secret_iv",
"users"."encrypted_otp_secret_salt",
"users"."otp_required_for_login",
"users"."otp_backup_codes",
"users"."public_email",
"users"."dashboard",
"users"."project_view",
"users"."consumed_timestep",
"users"."layout",
"users"."hide_project_limit",
"users"."note",
"users"."unlock_token",
"users"."otp_grace_period_started_at",
"users"."external",
"users"."incoming_email_token",
"users"."auditor",
"users"."require_two_factor_authentication_from_group",
"users"."two_factor_grace_period",
"users"."last_activity_on",
"users"."notified_of_own_activity",
"users"."preferred_language",
"users"."theme_id",
"users"."accepted_term_id",
"users"."feed_token",
"users"."private_profile",
"users"."roadmap_layout",
"users"."include_private_contributions",
"users"."commit_email",
"users"."group_view",
"users"."managing_group_id",
"users"."first_name",
"users"."last_name",
"users"."static_object_token",
"users"."user_type",
"users"."static_object_token_encrypted",
"users"."otp_secret_expires_at",
"users"."onboarding_in_progress",
"users"."color_mode_id",
"users"."composite_identity_enforced",
"users"."organization_id"
FROM "users"
WHERE "users"."id" IN (12,
11)https://postgres.ai/console/gitlab/gitlab-production-main/sessions/41858/commands/128431
Load user
SELECT "users"."id",
"users"."email",
"users"."encrypted_password",
"users"."reset_password_token",
"users"."reset_password_sent_at",
"users"."remember_created_at",
"users"."sign_in_count",
"users"."current_sign_in_at",
"users"."last_sign_in_at",
"users"."current_sign_in_ip",
"users"."last_sign_in_ip",
"users"."created_at",
"users"."updated_at",
"users"."name",
"users"."admin",
"users"."projects_limit",
"users"."failed_attempts",
"users"."locked_at",
"users"."username",
"users"."can_create_group",
"users"."can_create_team",
"users"."state",
"users"."color_scheme_id",
"users"."password_expires_at",
"users"."created_by_id",
"users"."last_credential_check_at",
"users"."avatar",
"users"."confirmation_token",
"users"."confirmed_at",
"users"."confirmation_sent_at",
"users"."unconfirmed_email",
"users"."hide_no_ssh_key",
"users"."admin_email_unsubscribed_at",
"users"."notification_email",
"users"."hide_no_password",
"users"."password_automatically_set",
"users"."encrypted_otp_secret",
"users"."encrypted_otp_secret_iv",
"users"."encrypted_otp_secret_salt",
"users"."otp_required_for_login",
"users"."otp_backup_codes",
"users"."public_email",
"users"."dashboard",
"users"."project_view",
"users"."consumed_timestep",
"users"."layout",
"users"."hide_project_limit",
"users"."note",
"users"."unlock_token",
"users"."otp_grace_period_started_at",
"users"."external",
"users"."incoming_email_token",
"users"."auditor",
"users"."require_two_factor_authentication_from_group",
"users"."two_factor_grace_period",
"users"."last_activity_on",
"users"."notified_of_own_activity",
"users"."preferred_language",
"users"."theme_id",
"users"."accepted_term_id",
"users"."feed_token",
"users"."private_profile",
"users"."roadmap_layout",
"users"."include_private_contributions",
"users"."commit_email",
"users"."group_view",
"users"."managing_group_id",
"users"."first_name",
"users"."last_name",
"users"."static_object_token",
"users"."user_type",
"users"."static_object_token_encrypted",
"users"."otp_secret_expires_at",
"users"."onboarding_in_progress",
"users"."color_mode_id",
"users"."composite_identity_enforced",
"users"."organization_id"
FROM "users"
INNER JOIN "organization_users" ON "organization_users"."user_id" = "users"."id"
WHERE "users"."user_type" = 11
AND "organization_users"."organization_id" = 1
ORDER BY "users"."id" ASC LIMIT 1https://postgres.ai/console/gitlab/gitlab-production-main/sessions/41858/commands/128432
This N+1 user load appears to match the existing behavior.
How to set up and validate locally
- Ensure the Organizations feature flags are enabled:
:allow_organization_creation:organization_switching:ui_for_organizations:organization_users_internal- Create a new Organization - https://gdk.test:3443/admin/organizations
- Create a group within the new Organization
- Create a new user within the new group
- Ensure group and project deletion protection is enabled: https://docs.gitlab.com/administration/settings/visibility_and_access_controls/#delayed-project-deletion
- Delete the group within the new Organization (as a non-admin user, from the groups page not the Admin page, to ensure the group retention period is followed)
- Ban the user that deleted the group, so that the cron worker will restore the group instead of fully delete it
- Wait for the deletion cron worker to restore the group. Or run it manually via Sidekiq or the Rails console
AdjournedGroupDeletionWorker.new.perform
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #442780 (closed)
Edited by Adil Farrukh