Skip to content

Finalize and fix batched background migrations MigrateVulnerabilitiesFeedbackToVulnerabilitiesStateTransition and CreateVulnerabilityLinks

Problem

It seems these migrations were both last run many months ago in:

  1. !111628 (merged)
  2. !107367 (merged)

But these have never been finalized following https://docs.gitlab.com/ee/development/database/batched_background_migrations.html

This is blocking some work in !135691 (closed)

I also noticed in implementing that MR there are several problems with these migrations and specs:

  1. The specs don't lock the schema which meant there was some drift over time and I need to add the schema locking in !135691 (closed) to get past problems with the new not-null constraint
  2. This above missing schema actually masks a problem with a tidy up that happened in !134380 (merged) because these columns have since been changed to uuid but when they were added they were varchar. But this is a problem for self-managed customers because they will get typecast errors like I was seeing like
    ERROR:  operator does not exist: character varying = uuid
      #   LINE 1: ...erability_occurrences WHERE vulnerability_occurrences.uuid =
      #                                                                         ^
      #   HINT:  No operator matches the given name and argument types. You might need to add explicit type casts.
      #   ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
  3. Once I reverted the changes in !134380 (merged) I ran into yet another problem where these migrations are depending on Vulnerabilities::CreateService which has changed since they were created. This again might very well mean these migrations do not work for self-managed customers where they have not been run yet
      ActiveModel::UnknownAttributeError:
        unknown attribute 'finding_id' for Vulnerability.
    
                  raise UnknownAttributeError.new(self, k.to_s)
                  ^^^^^
      # ./gems/activerecord-gitlab/lib/active_record/gitlab_patches/rescue_from.rb:40:in `_assign_attributes'
      # ./ee/app/services/vulnerabilities/create_service.rb:53:in `save_vulnerability'
      # ./ee/app/services/vulnerabilities/create_service.rb:36:in `block in execute'

Solution

  1. Figure out how to fix the changes to the migration so that they can successfully run if someone upgrades via a supported upgrade path
  2. Finalize the migrations
  3. Clean up the migrations somehow so that we can add a new not-null column in !135691 (closed) without breaking the specs. Ideally this would be deleting the migration but we need to wait for a required stop upgrade point to delete them
Edited by 🤖 GitLab Bot 🤖