Fully shard bulk_import_batch_trackers

To properly shard the bulk_import_batch_trackers table, please choose the one classification below that best fits its data and purpose. This choice will determine the table's database schema and sharding key.

After selecting the best option, apply the corresponding configuration by adding desired_sharding_key. Please note that you should also add desired_sharding_key_migration_job_name when you start the backfill migration for the sharding key.

1. Choose a Classification

Select the most suitable option for the bulk_import_batch_trackers table from the following:

A. Org-level with sharding key

This is for data that belongs to an entire organization.

  • Action:
    • Set gitlab_schema: gitlab_main_cell.
    • Add the sharding_key or desired_sharding_key.
    • If the key doesn't yet meet not-null or foreign key requirements, you can add a temporary exception. Please link to a follow-up issue in a comment next to the exception.
    • See an example at db/docs/epics.yml.

B. Cell-local

This is for data related to cell-specific features.

  • Action:
    • Set gitlab_schema: gitlab_main_cell_local.
    • Ensure there are no foreign key references to or from organization-level tables.

C. User-related

This is for data that is strictly and exclusively owned by a user.

  • Action:
    • Set gitlab_schema: gitlab_main_user.
    • Confirm the table's data belongs only to users.

D. Cell settings

This is for configuration settings that are specific to a single cell.

  • Action:
    • Set gitlab_schema: gitlab_main_cell_setting.
    • Ensure there are no foreign key references to or from organization-level tables.

Note: gitlab_main_clusterwide has been deprecated. If you believe this is the correct classification, please consult the Tenant Scale group for alternatives.

2. Add Migration Job Name

Important: When creating the backfill migration for this table, you must also add the desired_sharding_key_migration_job_name attribute to the database dictionary file (db/docs/bulk_import_batch_trackers.yml) in the same MR as the backfill migration.

The desired_sharding_key_migration_job_name should match the name of your batched background migration job class (e.g., BackfillBulkImportBatchTrackersProjectId).

Example:

desired_sharding_key_migration_job_name: BackfillBulkImportBatchTrackersProjectId

This attribute is required for proper tracking and monitoring of the sharding backfill process.

3. Complete the Task

  1. Add this issue to the epic: &11670
  2. Review the documentation below for more details.

Documentation

Edited by 🤖 GitLab Bot 🤖