Cells Iteration: [docs](https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/cells/#cells-iterations) This is not a blocker for Cells 1.0 (complete working feature deployed in Gitlab.com SaaS) or even Cells 1.5 (migrates existing customers to new cells). We need to convert all integer IDs in the primary cell to bigint to allow moving organizations from secondary cells to the primary. which will be done only in Cell 2.0. But based on the [initial investigation](https://gitlab.com/gitlab-org/core-platform-section/data-stores/-/issues/102#note_1885843420 "Analysis of solutions to cells cluster-wide unique sequences"), it was estimated at 8-12 months for converting all IDs in gitlab.com instance. So it's good to have this running alongside to be ahead of time. ### Things to consider 1. The major time taking task in the conversion process is the backfilling of the new bigint column. It runs for months for large tables (in billions), so we have to find better ways to tackle this. * If there is any backfilling on these table, we can make use of the table scans. * https://gitlab.com/gitlab-org/gitlab/-/issues/482470+ makes sure we initialize and backfill all integer IDs instead of a handling only a subset of IDs. * Recently there was a migration (https://gitlab.com/gitlab-org/gitlab/-/merge_requests/167972) to backfill sharding key on issues, and we made changes to initialize and backfill the int IDs in issues. * We can write a DangerBot to ping the DRIs in all Backfill migrations to decide whether we can include bigint IDs initialization in the MR. This is a boring solution, as it might be slightly tricky to automate this. 2. Also there is a huge scope in automating the [int -\> bigint](https://docs.gitlab.com/ee/development/database/avoiding_downtime_in_migrations.html#migrating-integer-primary-keys-to-bigint) process with gitlab-housekeeper. 3. Have to analyse how much load this would put on the database/CPU if we bulk convert IDs, if needed we can setup additional metrics to [throttle](https://docs.gitlab.com/ee/development/database/batched_background_migrations.html#throttling-batched-migrations) the BBMs. ### Integer IDs to be converted | Table | Column | Status | Feature Category | Related MRs | ETA | Backfill progress | |-------|--------|--------|------------------|-------------|-----|-------------------| | abuse_reports | assignee_id | Pending | | | | | | abuse_reports | id | Pending | | | | | | abuse_reports | reporter_id | Pending | | | | | | abuse_reports | resolved_by_id | Pending | | | | | | abuse_reports | user_id | Pending | | | | | | alert_management_alert_user_mentions | mentioned_groups_ids | Pending | | | | | | alert_management_alert_user_mentions | mentioned_projects_ids | Pending | | | | | | alert_management_alert_user_mentions | mentioned_users_ids | Pending | | | | | | alert_management_alerts | environment_id | Pending | | | | | | alert_management_alerts | prometheus_alert_id | Pending | | | | | | allowed_email_domains | group_id | Pending | | | | | | analytics_cycle_analytics_aggregations | last_full_issues_id | Pending | | | | | | analytics_cycle_analytics_aggregations | last_full_merge_requests_id | Pending | | | | | | analytics_cycle_analytics_aggregations | last_incremental_issues_id | Pending | | | | | | analytics_cycle_analytics_aggregations | last_incremental_merge_requests_id | Pending | | | | | | analytics_devops_adoption_segments | display_namespace_id | Pending | | | | | | analytics_devops_adoption_segments | namespace_id | Pending | | | | | | analytics_devops_adoption_snapshots | namespace_id | Pending | | | | | | appearances | id | Pending | | | | | | application_setting_terms | id | Pending | | | | | | application_settings | custom_project_templates_group_id | Pending | | | | | | application_settings | file_template_project_id | Pending | | | | | | application_settings | id | Pending | | | | | | application_settings | performance_bar_allowed_group_id | Pending | | | | | | application_settings | usage_stats_set_by_user_id | Pending | | | | | | application_settings | web_ide_oauth_application_id | Pending | | | | | | approval_merge_request_rules | merge_request_id | Pending | | | | | | approval_merge_request_rules_approved_approvers | user_id | Pending | | | | | | approval_merge_request_rules_groups | group_id | Pending | | | | | | approval_merge_request_rules_users | user_id | Pending | | | | | | approval_project_rules | project_id | Pending | | | | | | approval_project_rules_groups | group_id | Pending | | | | | | approval_project_rules_users | user_id | Pending | | | | | | approvals | id | Pending | | | | | | approvals | merge_request_id | Pending | | | | | | approvals | user_id | Pending | | | | | | approver_groups | group_id | Pending | | | | | | approver_groups | id | Pending | | | | | | approver_groups | target_id | Pending | | | | | | approvers | id | Pending | | | | | | approvers | target_id | Pending | | | | | | approvers | user_id | Pending | | | | | | audit_events | author_id | Pending | | | | | | audit_events | entity_id | Pending | | | | | | award_emoji | id | Pending | | | | | | award_emoji | user_id | Pending | | | | | | aws_roles | user_id | Pending | | | | | | badges | group_id | Pending | | | | | | badges | id | Pending | | | | | | badges | project_id | Pending | | | | | | board_assignees | assignee_id | Pending | | | | | | board_assignees | board_id | Pending | | | | | | board_assignees | id | Pending | | | | | | board_group_recent_visits | board_id | Pending | | | | | | board_group_recent_visits | group_id | Pending | | | | | | board_group_recent_visits | user_id | Pending | | | | | | board_labels | board_id | Pending | | | | | | board_labels | id | Pending | | | | | | board_labels | label_id | Pending | | | | | | board_project_recent_visits | board_id | Pending | | | | | | board_project_recent_visits | project_id | Pending | | | | | | board_project_recent_visits | user_id | Pending | | | | | | boards | group_id | Pending | | | | | | boards | id | Pending | | | | | | boards | milestone_id | Pending | | | | | | boards | project_id | Pending | | | | | | broadcast_messages | id | Pending | | | | | | bulk_import_configurations | bulk_import_id | Pending | | | | | | bulk_imports | user_id | Pending | | | | | | chat_names | id | Pending | | | | | | chat_names | user_id | Pending | | | | | | chat_teams | id | Pending | | | | | | chat_teams | namespace_id | Pending | | | | | | ci_group_variables | group_id | Pending | | | | | | ci_group_variables | id | Pending | | | | | | ci_namespace_mirrors | namespace_id | Pending | | | | | | ci_namespace_mirrors | traversal_ids | Pending | | | | | | ci_pending_builds | namespace_traversal_ids | Pending | | | | | | ci_pending_builds | plan_id | Pending | | | | | | ci_pending_builds | tag_ids | Pending | | | | | | ci_pipeline_chat_data | chat_name_id | Pending | | | | | | ci_pipeline_schedule_variables | id | Pending | | | | | | ci_pipeline_schedule_variables | pipeline_schedule_id | Pending | | | | | | ci_pipeline_schedules | id | Pending | | | | | | ci_pipeline_schedules | owner_id | Pending | | | | | | ci_pipeline_schedules | project_id | Pending | | | | | | ci_project_mirrors | namespace_id | Pending | | | | | | ci_project_mirrors | project_id | Pending | | | | | | ci_runner_namespaces | id | Pending | | | | | | ci_runner_namespaces | namespace_id | Pending | | | | | | ci_runner_namespaces | runner_id | Pending | | | | | | ci_runner_projects | id | Pending | | | | | | ci_runner_projects | project_id | Pending | | | | | | ci_runner_projects | runner_id | Pending | | | | | | ci_runners | allowed_plan_ids | Pending | | | | | | ci_runners | id | Pending | | | | | | ci_sources_pipelines | id | Pending | | | | | | ci_sources_pipelines | project_id | Pending | | | | | | ci_sources_pipelines | source_project_id | Pending | | | | | | ci_trigger_requests | commit_id | Pending | | | | | | ci_trigger_requests | id | Pending | | | | | | ci_trigger_requests | trigger_id | Pending | | | | | | ci_triggers | id | Pending | | | | | | ci_triggers | owner_id | Pending | | | | | | ci_triggers | project_id | Pending | | | | | | ci_variables | id | Pending | | | | | | ci_variables | project_id | Pending | | | | | | cluster_groups | cluster_id | Pending | | | | | | cluster_groups | group_id | Pending | | | | | | cluster_groups | id | Pending | | | | | | cluster_platforms_kubernetes | cluster_id | Pending | | | | | | cluster_platforms_kubernetes | id | Pending | | | | | | cluster_projects | cluster_id | Pending | | | | | | cluster_projects | id | Pending | | | | | | cluster_projects | project_id | Pending | | | | | | cluster_providers_gcp | cluster_id | Pending | | | | | | cluster_providers_gcp | id | Pending | | | | | | clusters | id | Pending | | | | | | clusters | management_project_id | Pending | | | | | | clusters | user_id | Pending | | | | | | clusters_kubernetes_namespaces | cluster_id | Pending | | | | | | clusters_kubernetes_namespaces | cluster_project_id | Pending | | | | | | clusters_kubernetes_namespaces | project_id | Pending | | | | | | commit_user_mentions | mentioned_groups_ids | Pending | | | | | | commit_user_mentions | mentioned_projects_ids | Pending | | | | | | commit_user_mentions | mentioned_users_ids | Pending | | | | | | compliance_management_frameworks | namespace_id | Pending | | | | | | container_repositories | id | Pending | | | | | | container_repositories | project_id | Pending | | | | | | conversational_development_index_metrics | id | Pending | | | | | | dast_scanner_profiles | project_id | Pending | | | | | | dependency_proxy_blobs | group_id | Pending | | | | | | dependency_proxy_blobs | id | Pending | | | | | | dependency_proxy_group_settings | group_id | Pending | | | | | | dependency_proxy_group_settings | id | Pending | | | | | | deploy_keys_projects | deploy_key_id | Pending | | | | | | deploy_keys_projects | id | Pending | | | | | | deploy_keys_projects | project_id | Pending | | | | | | deploy_tokens | id | Pending | | | | | | deployment_clusters | cluster_id | Pending | | | | | | deployment_clusters | deployment_id | Pending | | | | | | deployment_merge_requests | deployment_id | Pending | | | | | | deployment_merge_requests | environment_id | Pending | | | | | | deployment_merge_requests | merge_request_id | Pending | | | | | | deployments | environment_id | Pending | | | | | | deployments | id | Pending | | | | | | deployments | project_id | Pending | | | | | | deployments | user_id | Pending | | | | | | description_versions | epic_id | Pending | | | | | | description_versions | issue_id | Pending | | | | | | description_versions | merge_request_id | Pending | | | | | | design_management_designs | issue_id | Pending | | | | | | design_management_designs | project_id | Pending | | | | | | design_management_versions | author_id | Pending | | | | | | design_user_mentions | design_id | Pending | | | | | | design_user_mentions | mentioned_groups_ids | Pending | | | | | | design_user_mentions | mentioned_projects_ids | Pending | | | | | | design_user_mentions | mentioned_users_ids | Pending | | | | | | draft_notes | author_id | Pending | | | | | | draft_notes | merge_request_id | Pending | | | | | | elasticsearch_indexed_namespaces | namespace_id | Pending | | | | | | elasticsearch_indexed_projects | project_id | Pending | | | | | | emails | id | Pending | | | | | | emails | user_id | Pending | | | | | | environments | id | Pending | | | | | | environments | project_id | Pending | | | | | | epic_issues | epic_id | Pending | | | | | | epic_issues | id | Pending | | | | | | epic_issues | issue_id | Pending | | | | | | epic_metrics | epic_id | Pending | | | | | | epic_metrics | id | Pending | | | | | | epic_user_mentions | epic_id | Pending | | | | | | epic_user_mentions | mentioned_groups_ids | Pending | | | | | | epic_user_mentions | mentioned_projects_ids | Pending | | | | | | epic_user_mentions | mentioned_users_ids | Pending | | | | | | epics | assignee_id | Pending | | | | | | epics | author_id | Pending | | | | | | epics | closed_by_id | Pending | | | | | | epics | due_date_sourcing_epic_id | Pending | | | | | | epics | due_date_sourcing_milestone_id | Pending | | | | | | epics | group_id | Pending | | | | | | epics | id | Pending | | | | | | epics | issue_id | Pending | | | | | | epics | last_edited_by_id | Pending | | | | | | epics | parent_id | Pending | | | | | | epics | start_date_sourcing_epic_id | Pending | | | | | | epics | start_date_sourcing_milestone_id | Pending | | | | | | epics | updated_by_id | Pending | | | | | | events | author_id | Pending | | | | | | events | project_id | Pending | | | | | | feature_gates | id | Pending | | | | | | features | id | Pending | | | | | | fork_network_members | fork_network_id | Pending | | | | | | fork_network_members | forked_from_project_id | Pending | | | | | | fork_network_members | id | Pending | | | | | | fork_network_members | project_id | Pending | | | | | | fork_networks | id | Pending | | | | | | fork_networks | root_project_id | Pending | | | | | | geo_node_namespace_links | geo_node_id | Pending | | | | | | geo_node_namespace_links | id | Pending | | | | | | geo_node_namespace_links | namespace_id | Pending | | | | | | geo_node_statuses | geo_node_id | Pending | | | | | | geo_node_statuses | id | Pending | | | | | | geo_nodes | id | Pending | | | | | | geo_nodes | oauth_application_id | Pending | | | | | | gitlab_subscription_histories | hosted_plan_id | Pending | | | | | | gitlab_subscription_histories | namespace_id | Pending | | | | | | gitlab_subscriptions | hosted_plan_id | Pending | | | | | | gitlab_subscriptions | namespace_id | Pending | | | | | | gpg_key_subkeys | gpg_key_id | Pending | | | | | | gpg_key_subkeys | id | Pending | | | | | | gpg_keys | id | Pending | | | | | | gpg_keys | user_id | Pending | | | | | | gpg_signatures | gpg_key_id | Pending | | | | | | gpg_signatures | gpg_key_subkey_id | Pending | | | | | | gpg_signatures | id | Pending | | | | | | gpg_signatures | project_id | Pending | | | | | | group_custom_attributes | group_id | Pending | | | | | | group_custom_attributes | id | Pending | | | | | | historical_data | id | Pending | | | | | | identities | id | Pending | | | | | | identities | saml_provider_id | Pending | | | | | | identities | user_id | Pending | | | | | | import_export_uploads | id | Pending | | | | | | import_export_uploads | project_id | Pending | | | | | | import_failures | group_id | Pending | | | | | | index_statuses | id | Pending | | | | | | index_statuses | project_id | Pending | | | | | | insights | id | Pending | | | | | | insights | namespace_id | Pending | | | | | | insights | project_id | Pending | | | | | | integrations | id | Pending | | | | | | integrations | project_id | Pending | | | | | | internal_ids | namespace_id | Pending | | | | | | internal_ids | project_id | Pending | | | | | | ip_restrictions | group_id | Pending | | | | | | issue_assignees | issue_id | Pending | | | | | | issue_assignees | user_id | Pending | | | | | | issue_links | id | Pending | | | | | | issue_links | source_id | Pending | | | | | | issue_links | target_id | Pending | | | | | | issue_metrics | id | Pending | | | | | | issue_metrics | issue_id | Pending | | | | | | issue_tracker_data | integration_id | Pending | | | | | | issue_user_mentions | issue_id | Pending | | | | | | issue_user_mentions | mentioned_groups_ids | Pending | | | | | | issue_user_mentions | mentioned_projects_ids | Pending | | | | | | issue_user_mentions | mentioned_users_ids | Pending | | | | | | issues | author_id | Pending | | | | | | issues | closed_by_id | Pending | | | | | | issues | duplicated_to_id | Pending | | | | | | issues | id | Pending | | | | | | issues | last_edited_by_id | Pending | | | | | | issues | milestone_id | Pending | | | | | | issues | moved_to_id | Pending | | | | | | issues | project_id | Pending | | | | | | issues | promoted_to_epic_id | Pending | | | | | | issues | updated_by_id | Pending | | | | | | jira_connect_subscriptions | namespace_id | Pending | | | | | | jira_tracker_data | integration_id | Pending | | | | | | keys | id | Pending | | | | | | keys | user_id | Pending | | | | | | label_links | id | Pending | | | | | | label_links | label_id | Pending | | | | | | label_links | target_id | Pending | | | | | | label_priorities | id | Pending | | | | | | label_priorities | label_id | Pending | | | | | | label_priorities | project_id | Pending | | | | | | labels | group_id | Pending | | | | | | labels | id | Pending | | | | | | labels | project_id | Pending | | | | | | ldap_group_links | group_id | Pending | | | | | | ldap_group_links | id | Pending | | | | | | lfs_file_locks | id | Pending | | | | | | lfs_file_locks | project_id | Pending | | | | | | lfs_file_locks | user_id | Pending | | | | | | lfs_objects | id | Pending | | | | | | lfs_objects_projects | id | Pending | | | | | | lfs_objects_projects | lfs_object_id | Pending | | | | | | lfs_objects_projects | project_id | Pending | | | | | | licenses | id | Pending | | | | | | lists | board_id | Pending | | | | | | lists | id | Pending | | | | | | lists | label_id | Pending | | | | | | lists | milestone_id | Pending | | | | | | lists | user_id | Pending | | | | | | members | created_by_id | Pending | | | | | | members | id | Pending | | | | | | members | source_id | Pending | | | | | | members | user_id | Pending | | | | | | merge_request_assignees | merge_request_id | Pending | | | | | | merge_request_assignees | user_id | Pending | | | | | | merge_request_blocks | blocked_merge_request_id | Pending | | | | | | merge_request_blocks | blocking_merge_request_id | Pending | | | | | | merge_request_diff_commits | merge_request_diff_id | Pending | | | | | | merge_request_diff_files | merge_request_diff_id | Pending | | | | | | merge_request_diffs | id | Pending | | | | | | merge_request_diffs | merge_request_id | Pending | | | | | | merge_request_metrics | latest_closed_by_id | Pending | | | | | | merge_request_metrics | merge_request_id | Pending | | | | | | merge_request_metrics | merged_by_id | Pending | | | | | | merge_request_metrics | target_project_id | Pending | | | | | | merge_request_user_mentions | mentioned_groups_ids | Pending | | | | | | merge_request_user_mentions | mentioned_projects_ids | Pending | | | | | | merge_request_user_mentions | mentioned_users_ids | Pending | | | | | | merge_request_user_mentions | merge_request_id | Pending | | | | | | merge_requests | assignee_id | Pending | | | | | | merge_requests | author_id | Pending | | | | | | merge_requests | id | Pending | | | | | | merge_requests | last_edited_by_id | Pending | | | | | | merge_requests | latest_merge_request_diff_id | Pending | | | | | | merge_requests | merge_user_id | Pending | | | | | | merge_requests | milestone_id | Pending | | | | | | merge_requests | source_project_id | Pending | | | | | | merge_requests | target_project_id | Pending | | | | | | merge_requests | updated_by_id | Pending | | | | | | merge_requests_closing_issues | id | Pending | | | | | | merge_requests_closing_issues | issue_id | Pending | | | | | | merge_requests_closing_issues | merge_request_id | Pending | | | | | | merge_requests_compliance_violations | target_project_id | Pending | | | | | | merge_trains | merge_request_id | Pending | | | | | | merge_trains | target_project_id | Pending | | | | | | merge_trains | user_id | Pending | | | | | | milestones | group_id | Pending | | | | | | milestones | id | Pending | | | | | | milestones | project_id | Pending | | | | | | ml_models | user_id | Pending | | | | | | namespace_aggregation_schedules | namespace_id | Pending | | | | | | namespace_limits | namespace_id | Pending | | | | | | namespace_root_storage_statistics | namespace_id | Pending | | | | | | namespace_settings | namespace_id | Pending | | | | | | namespace_statistics | id | Pending | | | | | | namespace_statistics | namespace_id | Pending | | | | | | namespaces | custom_project_templates_group_id | Pending | | | | | | namespaces | file_template_project_id | Pending | | | | | | namespaces | id | Pending | | | | | | namespaces | owner_id | Pending | | | | | | namespaces | parent_id | Pending | | | | | | namespaces | traversal_ids | Pending | | | | | | note_diff_files | id | Pending | | | | | | notes | author_id | Pending | | | | | | notes | noteable_id | Pending | | | | | | notes | project_id | Pending | | | | | | notes | resolved_by_id | Pending | | | | | | notes | updated_by_id | Pending | | | | | | notification_settings | id | Pending | | | | | | notification_settings | source_id | Pending | | | | | | notification_settings | user_id | Pending | | | | | | oauth_access_grants | application_id | Pending | | | | | | oauth_access_grants | id | Pending | | | | | | oauth_access_grants | resource_owner_id | Pending | | | | | | oauth_access_tokens | application_id | Pending | | | | | | oauth_access_tokens | id | Pending | | | | | | oauth_access_tokens | resource_owner_id | Pending | | | | | | oauth_applications | id | Pending | | | | | | oauth_applications | owner_id | Pending | | | | | | oauth_openid_requests | access_grant_id | Pending | | | | | | oauth_openid_requests | id | Pending | | | | | | operations_feature_flags | project_id | Pending | | | | | | operations_feature_flags_clients | project_id | Pending | | | | | | p_ci_builds_metadata | project_id | Pending | | | | | | p_ci_job_artifacts | project_id | Pending | | | | | | p_ci_pipelines | merge_request_id | Pending | | | | | | p_ci_pipelines | pipeline_schedule_id | Pending | | | | | | p_ci_pipelines | project_id | Pending | | | | | | p_ci_pipelines | user_id | Pending | | | | | | p_ci_stages | project_id | Pending | | | | | | packages_build_infos | package_id | Pending | | | | | | packages_composer_cache_files | namespace_id | Pending | | | | | | packages_packages | creator_id | Pending | | | | | | packages_packages | project_id | Pending | | | | | | packages_tags | package_id | Pending | | | | | | pages_domain_acme_orders | pages_domain_id | Pending | | | | | | pages_domains | id | Pending | | | | | | pages_domains | project_id | Pending | | | | | | path_locks | id | Pending | | | | | | path_locks | project_id | Pending | | | | | | path_locks | user_id | Pending | | | | | | personal_access_tokens | id | Pending | | | | | | personal_access_tokens | user_id | Pending | | | | | | plans | id | Pending | | | | | | pool_repositories | shard_id | Pending | | | | | | pool_repositories | source_project_id | Pending | | | | | | programming_languages | id | Pending | | | | | | project_alerting_settings | project_id | Pending | | | | | | project_aliases | project_id | Pending | | | | | | project_authorizations | project_id | Pending | | | | | | project_authorizations | user_id | Pending | | | | | | project_auto_devops | id | Pending | | | | | | project_auto_devops | project_id | Pending | | | | | | project_ci_cd_settings | id | Pending | | | | | | project_ci_cd_settings | project_id | Pending | | | | | | project_custom_attributes | id | Pending | | | | | | project_custom_attributes | project_id | Pending | | | | | | project_daily_statistics | project_id | Pending | | | | | | project_deploy_tokens | deploy_token_id | Pending | | | | | | project_deploy_tokens | id | Pending | | | | | | project_deploy_tokens | project_id | Pending | | | | | | project_error_tracking_settings | project_id | Pending | | | | | | project_feature_usages | project_id | Pending | | | | | | project_features | id | Pending | | | | | | project_features | project_id | Pending | | | | | | project_group_links | group_id | Pending | | | | | | project_group_links | id | Pending | | | | | | project_group_links | project_id | Pending | | | | | | project_import_data | id | Pending | | | | | | project_import_data | project_id | Pending | | | | | | project_incident_management_settings | project_id | Pending | | | | | | project_metrics_settings | project_id | Pending | | | | | | project_mirror_data | id | Pending | | | | | | project_mirror_data | project_id | Pending | | | | | | project_repositories | project_id | Pending | | | | | | project_repositories | shard_id | Pending | | | | | | project_settings | project_id | Pending | | | | | | project_statistics | id | Pending | | | | | | project_statistics | namespace_id | Pending | | | | | | project_statistics | project_id | Pending | | | | | | projects | creator_id | Pending | | | | | | projects | id | Pending | | | | | | projects | marked_for_deletion_by_user_id | Pending | | | | | | projects | mirror_user_id | Pending | | | | | | projects | namespace_id | Pending | | | | | | prometheus_alert_events | project_id | Pending | | | | | | prometheus_alert_events | prometheus_alert_id | Pending | | | | | | prometheus_alerts | environment_id | Pending | | | | | | prometheus_alerts | id | Pending | | | | | | prometheus_alerts | project_id | Pending | | | | | | prometheus_alerts | prometheus_metric_id | Pending | | | | | | prometheus_metrics | id | Pending | | | | | | prometheus_metrics | project_id | Pending | | | | | | protected_branch_merge_access_levels | group_id | Pending | | | | | | protected_branch_merge_access_levels | id | Pending | | | | | | protected_branch_merge_access_levels | protected_branch_id | Pending | | | | | | protected_branch_merge_access_levels | user_id | Pending | | | | | | protected_branch_push_access_levels | deploy_key_id | Pending | | | | | | protected_branch_push_access_levels | group_id | Pending | | | | | | protected_branch_push_access_levels | id | Pending | | | | | | protected_branch_push_access_levels | protected_branch_id | Pending | | | | | | protected_branch_push_access_levels | user_id | Pending | | | | | | protected_branch_unprotect_access_levels | group_id | Pending | | | | | | protected_branch_unprotect_access_levels | id | Pending | | | | | | protected_branch_unprotect_access_levels | protected_branch_id | Pending | | | | | | protected_branch_unprotect_access_levels | user_id | Pending | | | | | | protected_branches | id | Pending | | | | | | protected_branches | project_id | Pending | | | | | | protected_environment_deploy_access_levels | group_id | Pending | | | | | | protected_environment_deploy_access_levels | id | Pending | | | | | | protected_environment_deploy_access_levels | protected_environment_id | Pending | | | | | | protected_environment_deploy_access_levels | user_id | Pending | | | | | | protected_environments | id | Pending | | | | | | protected_environments | project_id | Pending | | | | | | protected_tag_create_access_levels | deploy_key_id | Pending | | | | | | protected_tag_create_access_levels | group_id | Pending | | | | | | protected_tag_create_access_levels | id | Pending | | | | | | protected_tag_create_access_levels | protected_tag_id | Pending | | | | | | protected_tag_create_access_levels | user_id | Pending | | | | | | protected_tags | id | Pending | | | | | | protected_tags | project_id | Pending | | | | | | push_rules | id | Pending | | | | | | push_rules | project_id | Pending | | | | | | redirect_routes | id | Pending | | | | | | redirect_routes | source_id | Pending | | | | | | release_links | release_id | Pending | | | | | | releases | author_id | Pending | | | | | | releases | id | Pending | | | | | | releases | project_id | Pending | | | | | | remote_mirrors | id | Pending | | | | | | remote_mirrors | project_id | Pending | | | | | | repository_languages | programming_language_id | Pending | | | | | | repository_languages | project_id | Pending | | | | | | requirements | project_id | Pending | | | | | | resource_label_events | epic_id | Pending | | | | | | resource_label_events | issue_id | Pending | | | | | | resource_label_events | label_id | Pending | | | | | | resource_label_events | merge_request_id | Pending | | | | | | resource_label_events | user_id | Pending | | | | | | resource_state_events | epic_id | Pending | | | | | | reviews | author_id | Pending | | | | | | reviews | merge_request_id | Pending | | | | | | reviews | project_id | Pending | | | | | | routes | id | Pending | | | | | | routes | source_id | Pending | | | | | | saml_providers | group_id | Pending | | | | | | saml_providers | id | Pending | | | | | | scim_oauth_access_tokens | group_id | Pending | | | | | | scim_oauth_access_tokens | id | Pending | | | | | | sent_notifications | noteable_id | Pending | | | | | | sent_notifications | project_id | Pending | | | | | | sent_notifications | recipient_id | Pending | | | | | | shards | id | Pending | | | | | | slack_integrations | id | Pending | | | | | | slack_integrations | integration_id | Pending | | | | | | smartcard_identities | user_id | Pending | | | | | | snippet_user_mentions | mentioned_groups_ids | Pending | | | | | | snippet_user_mentions | mentioned_projects_ids | Pending | | | | | | snippet_user_mentions | mentioned_users_ids | Pending | | | | | | snippet_user_mentions | snippet_id | Pending | | | | | | snippets | author_id | Pending | | | | | | snippets | id | Pending | | | | | | snippets | project_id | Pending | | | | | | software_license_policies | id | Pending | | | | | | software_license_policies | project_id | Pending | | | | | | software_license_policies | software_license_id | Pending | | | | | | software_licenses | id | Pending | | | | | | spam_logs | id | Pending | | | | | | spam_logs | user_id | Pending | | | | | | sprints | iterations_cadence_id | Pending | | | | | | subscriptions | id | Pending | | | | | | subscriptions | project_id | Pending | | | | | | subscriptions | subscribable_id | Pending | | | | | | subscriptions | user_id | Pending | | | | | | taggings | tag_id | Pending | | | | | | taggings | tagger_id | Pending | | | | | | tags | id | Pending | | | | | | term_agreements | id | Pending | | | | | | term_agreements | term_id | Pending | | | | | | term_agreements | user_id | Pending | | | | | | timelogs | id | Pending | | | | | | timelogs | issue_id | Pending | | | | | | timelogs | merge_request_id | Pending | | | | | | timelogs | project_id | Pending | | | | | | timelogs | user_id | Pending | | | | | | todos | author_id | Pending | | | | | | todos | group_id | Pending | | | | | | todos | id | Pending | | | | | | todos | project_id | Pending | | | | | | todos | target_id | Pending | | | | | | todos | user_id | Pending | | | | | | trending_projects | id | Pending | | | | | | trending_projects | project_id | Pending | | | | | | uploads | id | Pending | | | | | | uploads | model_id | Pending | | | | | | user_agent_details | id | Pending | | | | | | user_agent_details | subject_id | Pending | | | | | | user_callouts | id | Pending | | | | | | user_callouts | user_id | Pending | | | | | | user_custom_attributes | id | Pending | | | | | | user_custom_attributes | user_id | Pending | | | | | | user_follow_users | followee_id | Pending | | | | | | user_follow_users | follower_id | Pending | | | | | | user_preferences | id | Pending | | | | | | user_preferences | user_id | Pending | | | | | | user_statuses | user_id | Pending | | | | | | user_synced_attributes_metadata | id | Pending | | | | | | user_synced_attributes_metadata | user_id | Pending | | | | | | users | accepted_term_id | Pending | | | | | | users | color_scheme_id | Pending | | | | | | users | created_by_id | Pending | | | | | | users | id | Pending | | | | | | users | managing_group_id | Pending | | | | | | users_ops_dashboard_projects | project_id | Pending | | | | | | users_ops_dashboard_projects | user_id | Pending | | | | | | users_star_projects | id | Pending | | | | | | users_star_projects | project_id | Pending | | | | | | users_star_projects | user_id | Pending | | | | | | vulnerability_exports | group_id | Pending | | | | | | vulnerability_feedback | author_id | Pending | | | | | | vulnerability_feedback | comment_author_id | Pending | | | | | | vulnerability_feedback | id | Pending | | | | | | vulnerability_feedback | issue_id | Pending | | | | | | vulnerability_feedback | merge_request_id | Pending | | | | | | vulnerability_feedback | project_id | Pending | | | | | | vulnerability_identifiers | project_id | Pending | | | | | | vulnerability_merge_request_links | merge_request_id | Pending | | | | | | vulnerability_occurrences | project_id | Pending | | | | | | vulnerability_scanners | project_id | Pending | | | | | | vulnerability_user_mentions | mentioned_groups_ids | Pending | | | | | | vulnerability_user_mentions | mentioned_projects_ids | Pending | | | | | | vulnerability_user_mentions | mentioned_users_ids | Pending | | | | | | web_hook_logs | web_hook_id | Pending | | | | | | web_hooks | group_id | Pending | | | | | | web_hooks | id | Pending | | | | | | web_hooks | integration_id | Pending | | | | | | web_hooks | project_id | Pending | | | | | | wiki_page_meta | id | Pending | | | | | | wiki_page_slugs | id | Pending | | | | | ### NOTE: The above has been taken from [db/integer_ids_not_yet_initialized_to_bigint.yml](https://gitlab.com/gitlab-org/gitlab/blob/fff3f8369cc87460f6fee80d62ed3516512a083d/db/integer_ids_not_yet_initialized_to_bigint.yml), it was the integer IDs from the schema before https://gitlab.com/gitlab-org/gitlab/-/merge_requests/145752. There is a small set of discrepancy between the above list and the integer IDs from current production. **Query to get all integer IDs:** ```pgsql SELECT table_name, column_name, data_type, column_default FROM information_schema.columns a WHERE a.table_schema = 'public' AND ( -- columns like "id" and "project_id" (a.data_type = 'integer' AND (a.column_name = 'id' OR a.column_name LIKE '%\_id')) OR -- columns like "traversal_ids" (a.data_type = 'ARRAY' AND a.udt_name = '_int4' AND a.column_name LIKE '%\_ids') ) AND NOT EXISTS ( -- skip columns when migration is in progress SELECT 1 FROM information_schema.columns b WHERE b.table_schema = a.table_schema AND b.table_name = a.table_name AND b.data_type = 'bigint' AND b.column_name = (a.column_name || '_convert_to_bigint') ) AND NOT EXISTS ( -- skip columns from partitioned tables SELECT 1 FROM postgres_partitions c WHERE c.schema = 'public' AND c.name = a.table_name ) ORDER BY table_schema, table_name, column_name; ``` ```ruby > in_schema = [] > file = YAML.load_file('db/integer_ids_not_yet_initialized_to_bigint.yml') > file.map { |k, v| v.each { |c| in_schema << "#{k}.#{c}" } } > in_schema.count => 576 # This is the total IDs in db/integer_ids_not_yet_initialized_to_bigint.yml # Populated in_db from the o/p (concatenated table_name.col_name) of above query in prod db > in_db.count => 572 > in_schema - in_db => ["prometheus_alert_events.project_id", "prometheus_alert_events.prometheus_alert_id", "prometheus_alerts.environment_id", "prometheus_alerts.id", "prometheus_alerts.project_id", "prometheus_alerts.prometheus_metric_id", "prometheus_metrics.id", "prometheus_metrics.project_id"] # prometheus_alert_events, prometheus_alerts, prometheus_metrics tables got deleted recently, so we can remove these from the integer_ids_not_yet_initialized_to_bigint.yml file > in_db - in_schema => ["merge_request_assignees.id", "personal_access_tokens_restored_from_backup.id", "personal_access_tokens_restored_from_backup.user_id", "temp_user_details_issue18240.gitlab_dotcom_id"] # personal_access_tokens_restored_from_backup, personal_access_tokens_restored_from_backup and temp_user_details_issue18240 looks like interim/temp tables # We have to see why `merge_request_assignees.id` is 'integer' in the Prod but not in schema before converting all int -> bigint in the schema ```