Skip to content

Domain verification should initiate Enterprise Users Automatic Claim

Bogdan Denkovych requested to merge bdenkovych-issue-388415 into master

What does this MR do and why?

Related to #388415 (closed)

This MR initiates Enterprise Users Automatic Claim for successful domain verification and re-verification.

This change is under enterprise_users_automatic_claim FF.

See #388415 (closed) description for details and for "why?".

How to set up and validate locally

  1. Familiarize with the Enterprise User definition to understand what context you should prepare in your GitLab instance for testing

  2. Enable enterprise_users_automatic_claim FF. Guide: https://docs.gitlab.com/ee/administration/feature_flags.html#enable-or-disable-the-feature

  3. Make sure the GitLab instance simulates or a SaaS instance since Enterprise Users is a SaaS feature

  4. Create a top-level paid group with Premium subscription as a minimum

  5. Create a few users on the GitLab instance with emails that have a domain that you can verify

  6. Verify the domain for the group, docs: https://docs.gitlab.com/ee/user/enterprise_user/#set-up-a-verified-domain

  7. Confirm that Groups::EnterpriseUsers::BulkAssociateByDomainWorker has been scheduled and completed after successful domain verification

  8. Confirm that this worker scheduled Groups::EnterpriseUsers::AssociateWorker for all users with email domain that is equal to the verified domain. Confirm their execution is complete

  9. Confirm that those users have received user_associated_with_enterprise_group_email

  10. Additionally, you can check logs. 'Associated the user with the enterprise group' log entries should be added with the context data, like user_id and group_id

  11. You can also check from the rails console User.find_by(email: 'EMAIL-HERE').user_detail.enterprise_group_id attribute. Its value should be the same as the group's id

  12. Go to the domain and manually retry verification: https://docs.gitlab.com/ee/user/project/pages/custom_domains_ssl_tls_certification/#4-verify-the-domains-ownership

  13. Confirm that Groups::EnterpriseUsers::BulkAssociateByDomainWorker has been scheduled and completed after successful domain verification

  14. Confirm that this worker didn't schedule Groups::EnterpriseUsers::AssociateWorker for users from previous steps since they had been marked as enterprise users of the group.

Since the Enterprise User definition consists of lots of items, we could reproduce lots of cases. You can read the specs https://gitlab.com/gitlab-org/gitlab/-/blob/af58ea0f4110b277120e8c45cbcc59d70c8a5ed9/ee/spec/services/groups/enterprise_users/associate_service_spec.rb (and ee/spec/workers/groups/enterprise_users/bulk_associate_by_domain_worker_spec.rb in this MR) for detail.

DB

Migration

bin/rails db:migrate 
bogdanvlviv@lenovo:~/gitlab-development-kit/gitlab$ bin/rails db:migrate RAILS_ENV=test
main: == [advisory_lock_connection] object_id: 221860, pg_backend_pid: 27819
main: == 20231009105056 IndexUsersOnEmailDomainAndId: migrating =====================
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0143s
main: -- index_exists?(:users, "LOWER(SPLIT_PART(email, '@', 2)), id", {:name=>"index_users_on_email_domain_and_id", :algorithm=>:concurrently})
main:    -> 0.0124s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0005s
main: -- add_index(:users, "LOWER(SPLIT_PART(email, '@', 2)), id", {:name=>"index_users_on_email_domain_and_id", :algorithm=>:concurrently})
main:    -> 0.0026s
main: -- execute("RESET statement_timeout")
main:    -> 0.0006s
main: == 20231009105056 IndexUsersOnEmailDomainAndId: migrated (0.0468s) ============

main: == [advisory_lock_connection] object_id: 221860, pg_backend_pid: 27819
ci: == [advisory_lock_connection] object_id: 222080, pg_backend_pid: 27821
ci: == 20231009105056 IndexUsersOnEmailDomainAndId: migrating =====================
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- view_exists?(:postgres_partitions)
ci:    -> 0.0009s
ci: -- index_exists?(:users, "LOWER(SPLIT_PART(email, '@', 2)), id", {:name=>"index_users_on_email_domain_and_id", :algorithm=>:concurrently})
ci:    -> 0.0136s
ci: -- execute("SET statement_timeout TO 0")
ci:    -> 0.0007s
ci: -- add_index(:users, "LOWER(SPLIT_PART(email, '@', 2)), id", {:name=>"index_users_on_email_domain_and_id", :algorithm=>:concurrently})
ci:    -> 0.0024s
ci: -- execute("RESET statement_timeout")
ci:    -> 0.0005s
ci: == 20231009105056 IndexUsersOnEmailDomainAndId: migrated (0.0343s) ============

ci: == [advisory_lock_connection] object_id: 222080, pg_backend_pid: 27821
bin/rails db:rollback 
bogdanvlviv@lenovo:~/gitlab-development-kit/gitlab$ bin/rails db:rollback:main RAILS_ENV=test
main: == [advisory_lock_connection] object_id: 221440, pg_backend_pid: 28194
main: == 20231009105056 IndexUsersOnEmailDomainAndId: reverting =====================
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0146s
main: -- indexes(:users)
main:    -> 0.0120s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0005s
main: -- remove_index(:users, {:algorithm=>:concurrently, :name=>"index_users_on_email_domain_and_id"})
main:    -> 0.0019s
main: -- execute("RESET statement_timeout")
main:    -> 0.0006s
main: == 20231009105056 IndexUsersOnEmailDomainAndId: reverted (0.0446s) ============

main: == [advisory_lock_connection] object_id: 221440, pg_backend_pid: 28194
bogdanvlviv@lenovo:~/gitlab-development-kit/gitlab$ bin/rails db:rollback:ci RAILS_ENV=test
ci: == [advisory_lock_connection] object_id: 221360, pg_backend_pid: 28542
ci: == 20231009105056 IndexUsersOnEmailDomainAndId: reverting =====================
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- view_exists?(:postgres_partitions)
ci:    -> 0.0598s
ci: -- indexes(:users)
ci:    -> 0.0578s
ci: -- execute("SET statement_timeout TO 0")
ci:    -> 0.0034s
ci: -- remove_index(:users, {:algorithm=>:concurrently, :name=>"index_users_on_email_domain_and_id"})
ci:    -> 0.0042s
ci: -- execute("RESET statement_timeout")
ci:    -> 0.0031s
ci: == 20231009105056 IndexUsersOnEmailDomainAndId: reverted (0.1787s) ============

ci: == [advisory_lock_connection] object_id: 221360, pg_backend_pid: 28542

Query Plans

⚠ NOTE: The query plans's results might not be accurate since it used somewhat random users' ids because I have no access to see Postgres.ai instance data to figure out which ids to use.

⚠ UPDATE ON THIS NOTE: The query plans use appropriate id ranges now as per suggestion !133218 (comment 1596375599)

Create index_users_on_email_domain_and_id index

CREATE INDEX index_users_on_email_domain_and_id ON users USING btree (lower(split_part((email)::text, '@'::text, 2)), id);

each_batch finds the lowest id for the query

SELECT "users"."id" FROM "users" WHERE (lower(split_part(email, '@', 2)) = 'gitlab.com') ORDER BY "users"."id" ASC LIMIT 1;

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/22998/commands/74087

each_batch finds the next id for the query based on the batch size configuration - 100.

SELECT "users"."id" FROM "users" WHERE (lower(split_part(email, '@', 2)) = 'gitlab.com') AND "users"."id" >= 1 ORDER BY "users"."id" ASC LIMIT 1 OFFSET 100;

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/22998/commands/74088

each_batch query with the id range for the batch

SELECT "users"."id" FROM "users" LEFT OUTER JOIN "user_details" ON "user_details"."user_id" = "users"."id" WHERE (lower(split_part(email, '@', 2)) = 'gitlab.com') AND "users"."id" >= 1 AND "users"."id" < 480804 AND (user_details.enterprise_group_id != 187 OR user_details.enterprise_group_id IS NULL);

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/22998/commands/74090

each_batch query with the id range for the last batch

SELECT "users"."id" FROM "users" LEFT OUTER JOIN "user_details" ON "user_details"."user_id" = "users"."id" WHERE (lower(split_part(email, '@', 2)) = 'gitlab.com') AND "users"."id" >= 15791577 AND (user_details.enterprise_group_id != 187 OR user_details.enterprise_group_id IS NULL);

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/22998/commands/74097

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Bogdan Denkovych

Merge request reports