Reindex Elastic/Zoekt data to fix Archived

What does this MR do and why?

This code adds a new database migration that will run in the background to update search indexes for archived GitLab groups. The migration specifically targets groups that have been archived and triggers workers to reindex their data in both Elasticsearch and Zoekt search systems. The migration only runs on GitLab.com and processes namespace settings in batches to avoid performance issues. It includes proper documentation, tests to verify the functionality works correctly, and follows GitLab's standard migration patterns for safely updating large amounts of data without disrupting the application.

References

Screenshots or screen recordings

Before After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Query report

https://console.postgres.ai/shared/a28a4439-e896-4d3a-9810-88f9e15aea9b

SELECT
    "namespace_settings"."created_at",
    "namespace_settings"."updated_at",
    "namespace_settings"."namespace_id",
    "namespace_settings"."prevent_forking_outside_group",
    "namespace_settings"."allow_mfa_for_subgroups",
    "namespace_settings"."default_branch_name",
    "namespace_settings"."repository_read_only",
    "namespace_settings"."resource_access_token_creation_allowed",
    "namespace_settings"."prevent_sharing_groups_outside_hierarchy",
    "namespace_settings"."new_user_signups_cap",
    "namespace_settings"."setup_for_company",
    "namespace_settings"."jobs_to_be_done",
    "namespace_settings"."runner_token_expiration_interval",
    "namespace_settings"."subgroup_runner_token_expiration_interval",
    "namespace_settings"."project_runner_token_expiration_interval",
    "namespace_settings"."show_diff_preview_in_email",
    "namespace_settings"."enabled_git_access_protocol",
    "namespace_settings"."unique_project_download_limit",
    "namespace_settings"."unique_project_download_limit_interval_in_seconds",
    "namespace_settings"."unique_project_download_limit_allowlist",
    "namespace_settings"."auto_ban_user_on_excessive_projects_download",
    "namespace_settings"."only_allow_merge_if_pipeline_succeeds",
    "namespace_settings"."allow_merge_on_skipped_pipeline",
    "namespace_settings"."only_allow_merge_if_all_discussions_are_resolved",
    "namespace_settings"."default_compliance_framework_id",
    "namespace_settings"."runner_registration_enabled",
    "namespace_settings"."allow_runner_registration_token",
    "namespace_settings"."unique_project_download_limit_alertlist",
    "namespace_settings"."emails_enabled",
    "namespace_settings"."experiment_features_enabled",
    "namespace_settings"."default_branch_protection_defaults",
    "namespace_settings"."service_access_tokens_expiration_enforced",
    "namespace_settings"."product_analytics_enabled",
    "namespace_settings"."allow_merge_without_pipeline",
    "namespace_settings"."enforce_ssh_certificates",
    "namespace_settings"."math_rendering_limits_enabled",
    "namespace_settings"."lock_math_rendering_limits_enabled",
    "namespace_settings"."duo_features_enabled",
    "namespace_settings"."lock_duo_features_enabled",
    "namespace_settings"."disable_personal_access_tokens",
    "namespace_settings"."early_access_program_participant",
    "namespace_settings"."remove_dormant_members",
    "namespace_settings"."remove_dormant_members_period",
    "namespace_settings"."early_access_program_joined_by_id",
    "namespace_settings"."seat_control",
    "namespace_settings"."last_dormant_member_review_at",
    "namespace_settings"."enterprise_users_extensions_marketplace_opt_in_status",
    "namespace_settings"."spp_repository_pipeline_access",
    "namespace_settings"."lock_spp_repository_pipeline_access",
    "namespace_settings"."archived",
    "namespace_settings"."resource_access_token_notify_inherited",
    "namespace_settings"."lock_resource_access_token_notify_inherited",
    "namespace_settings"."pipeline_variables_default_role",
    "namespace_settings"."force_pages_access_control",
    "namespace_settings"."extended_grat_expiry_webhooks_execute",
    "namespace_settings"."jwt_ci_cd_job_token_enabled",
    "namespace_settings"."jwt_ci_cd_job_token_opted_out",
    "namespace_settings"."require_dpop_for_manage_api_endpoints",
    "namespace_settings"."security_policies",
    "namespace_settings"."duo_nano_features_enabled",
    "namespace_settings"."disable_invite_members",
    "namespace_settings"."model_prompt_cache_enabled",
    "namespace_settings"."lock_model_prompt_cache_enabled",
    "namespace_settings"."web_based_commit_signing_enabled",
    "namespace_settings"."lock_web_based_commit_signing_enabled",
    "namespace_settings"."allow_enterprise_bypass_placeholder_confirmation",
    "namespace_settings"."enterprise_bypass_expires_at",
    "namespace_settings"."hide_email_on_profile",
    "namespace_settings"."allow_personal_snippets",
    "namespace_settings"."auto_duo_code_review_enabled",
    "namespace_settings"."lock_auto_duo_code_review_enabled",
    "namespace_settings"."step_up_auth_required_oauth_provider",
    "namespace_settings"."duo_remote_flows_enabled",
    "namespace_settings"."lock_duo_remote_flows_enabled",
    "namespace_settings"."duo_agent_platform_request_count",
    "namespace_settings"."disable_ssh_keys"
FROM
    "namespace_settings"
WHERE
    "namespace_settings"."archived" = TRUE

Related to #579515

Edited by Ravi Kumar

Merge request reports

Loading