There are too many exemptions in spec/lib/gitlab/database/sharding_key_spec.rb

Problem statement

All these exemptions (see below) are not inspiring confidence for the https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/1626+

  # Specific tables can be temporarily exempt from this requirement. You must add an issue link in a comment next to
  # the table name to remove this once a decision has been made.
  let(:allowed_to_be_missing_sharding_key) do
    ...
  end

  # Specific tables can be temporarily exempt from this requirement. You must add an issue link in a comment next to
  # the table name to remove this once a decision has been made.
  let(:allowed_to_be_missing_not_null) do
    ...
  end

  # The following tables are work in progress as part of
  # https://gitlab.com/gitlab-org/gitlab/-/issues/398199
  # TODO: Remove these excepttions once the issue is closed.
  let(:uploads_and_partitions) do
    ...
  end

  # Some reasons to exempt a table:
  #   1. It has no foreign key for performance reasons
  #   2. It does not yet have a foreign key as the index is still being backfilled
  let(:allowed_to_be_missing_foreign_key) do
    ...
  end

Related: https://gitlab.com/groups/gitlab-com/gl-infra/tenant-scale/-/wikis/Sharding-Priorities

Action item

Work undertaken

🏁 InternalIds (# rows: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/41950/commands/128699)
  • set NOT NULL (NOT VALID) on sharding key + prepare async constraint validation -> !199480 (merged)
  • verify the status of the BBM + verify table contains NOT NULL (NOT VALID) constraint
  • validate NOT NULL constraint on sharding key -> !199806 (merged)
  • add validate presence: true on the sharding columns in the rails model
  • update db/docs/*.yml with sharding key
  • verify the NOT NULL constraint check is set on the table's sharding key
🏁 CiTriggers (# rows: https://postgres.ai/console/gitlab/gitlab-production-ci/sessions/41954/commands/128698)
  • set NOT NULL on sharding key -> !199566 (merged)
  • add validate presence: true on the sharding columns in the rails model
  • update db/docs/*.yml with sharding key
  • verify the NOT NULL constraint check is set on the table's sharding key
🏁 CiPipelineSchedules (# rows: https://postgres.ai/console/gitlab/gitlab-production-ci/sessions/41954/commands/128697)
  • set NOT NULL on sharding key -> !199502 (merged)
  • add validate presence: true on the sharding columns in the rails model
  • update db/docs/*.yml with sharding key
  • verify the NOT NULL constraint check is set on the table's sharding key
🏁 GpgSignatures (# rows: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/42015/commands/129066)
  • set NOT NULL (NOT VALID) on sharding key + prepare async constraint validation -> !200250 (merged)
  • verify table contains NOT NULL (NOT VALID) constraint
  • verify table contains NOT NULL constraint (due to the prepare_async_check_constraint_validation)
  • validate NOT NULL constraint on sharding key -> !200252 (merged)
  • update db/docs/*.yml with sharding key
🏁 AnalyticsDevopsAdoptionSegments (# rows: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/42394/commands/129937)
🏁 CiSourcesPipelines (# rows: https://postgres.ai/console/gitlab/gitlab-production-ci/sessions/42471/commands/130010)
Edited by Tomasz Skorupa