Skip to content

Make organization_id column non nullable

Overview

Add a NOT NULL constraint to the organization_id column in the keys table to ensure data integrity and complete the sharding key implementation.

Context

The organization_id column was added to the keys table in !208105 (merged) as part of the sharding key implementation for GitLab Cells architecture. After backfilling all existing records (#577245) and updating application logic (#577243, #577244), we need to add a NOT NULL constraint to ensure all future records have this field populated.

Prerequisites

Before this issue can be started, the following must be completed:

Requirements

1. Verify Backfill Completion

Before adding the constraint:

  • Verify that 100% of records in the keys table have organization_id populated
  • Check for any NULL values: SELECT COUNT(*) FROM keys WHERE organization_id IS NULL
  • Investigate and resolve any remaining NULL records
  • Ensure application code is setting organization_id for all new records

2. Add NOT NULL Constraint

Create a database migration to:

  • Add NOT NULL constraint to the organization_id column
  • Use add_not_null_constraint or similar helper method
  • Ensure the migration is safe and can be rolled back if needed
  • Follow GitLab's database migration best practices

3. Update Model Validation

Update the Key model (app/models/key.rb):

  • Add validates :organization_id, presence: true validation
  • Remove any temporary validation logic that allowed NULL values
  • Ensure validation messages are clear and helpful

4. Update Database Documentation

Update db/docs/keys.yml:

  • Mark organization_id as required (not nullable)
  • Update any comments or documentation about the field
  • Ensure the sharding key configuration is correct

5. Testing

  • Test the migration on a staging environment
  • Verify that new keys cannot be created without organization_id
  • Test rollback scenario
  • Ensure no regressions in key creation/management

Implementation Steps

  1. Verify all prerequisites are met
  2. Create database migration to add NOT NULL constraint
  3. Update model validation
  4. Update database documentation
  5. Test on staging environment
  6. Deploy to production
  7. Monitor for any issues

Acceptance Criteria

  • All existing records have organization_id populated (verified)
  • NOT NULL constraint added to organization_id column
  • Model validation updated to require organization_id
  • Database documentation updated
  • Migration tested on staging
  • Migration deployed to production successfully
  • No regressions in key creation or management
  • Monitoring shows no errors related to the constraint

Migration Example

class AddNotNullConstraintToKeysOrganizationId < Gitlab::Database::Migration[2.3]
  disable_ddl_transaction!

  def up
    add_not_null_constraint :keys, :organization_id
  end

  def down
    remove_not_null_constraint :keys, :organization_id
  end
end

Rollback Plan

If issues arise after deployment:

  • The migration can be rolled back to remove the constraint
  • Application code should continue to work (it's already setting the field)
  • Investigate and fix any issues before re-attempting

References

Documentation

Edited by 🤖 GitLab Bot 🤖