Skip to content

XS Add sharding key YAML config to p_ci_finished_build_ch_sync_events

What we need to do

The p_ci_finished_build_ch_sync_events has build_id, so we can add project_id and use ci_builds.project_id to backfill p_ci_finished_build_ch_sync_events

How to do it

If we can get by with adding a LFK, that would be the easiest route. The lack of a partition_id shouldn't be a blocker, since the p_ci_builds table currently has only 3 partitions in .com.

Alternative

If we cannot use desired_sharding_key, we can:

  1. (!163926 (merged)) add a non-nullable project_id column (default -1) and an LFK pointing to the projects table. The reason for a non-nullable column is to avoid extra work later on. It shouldn't be an issue since these records are getting deleted frequently, therefore in at most 30 days we'll have a table with all valid project IDs.
  2. (!163926 (merged)) add a validates :project_id, presence: true, on: :create validation to ensure that the project_id field is present.
  3. (!164766 (merged)) after some days, merge an MR with a post-deployment migration truncating p_ci_finished_build_ch_sync_events if ClickHouse unconfigured. Since .com has ClickHouse configured, this won't have an impact there.
  4. (!166733 (merged)) in %17.5, once all records are valid, change the validation to work at all times.
  5. (!166736 (merged)) remove column default.
  6. (!166736 (merged)) Add sharding_key to db/docs/p_ci_finished_build_ch_sync_events.yml
Edited by Pedro Pombeiro