Skip to content

Data migration to set sharding key on jira_tracker_data table

In https://gitlab.com/gitlab-org/gitlab/-/issues/524675 we introduced project_id/group_id/organization_id on jira_tracker_data tables and setting it on every new instance integration going forward.

In order to set the sharding key on these tables we need to set sharding key on existing records.

Create a data migration that sets default sharding key on existing records if they don't already have one.

Latest Update

After running batched background migration on jira_tracker_data it looks like we have an application bug which doesn't set sharding key properly, as we had 2 records without sharding key after the backfill, then 7, then 64, as seen in #545940 (comment 2626175766)

After spending some time trying to find what could be causing this, we decided to add some logging to the Create/Update Propagation Services, since this is where we bulk insert the records skipping any ActiveRecord validations, in an attempt to narrow the issue down, as it's unclear where exactly this is happening.

Once the bug is found and fixed, we need to requeue batched background migration to fix the problematic records again, wait for the next milestone and mark batched background migration as finalized in order to set a not null constraint on the table (in scope of the other issue in the parent epic).

Edited by George Koltsov