Finalize and fix batched background migrations `MigrateVulnerabilitiesFeedbackToVulnerabilitiesStateTransition` and `CreateVulnerabilityLinks`
Problem
It seems these migrations were both last run many months ago in:
But these have never been finalized following https://docs.gitlab.com/ee/development/database/batched_background_migrations.html
This is blocking some work in !135691 (closed)
I also noticed in implementing that MR there are several problems with these migrations and specs:
- The specs don't lock the
schema
which meant there was some drift over time and I need to add the schema locking in !135691 (closed) to get past problems with the new not-null constraint - This above missing
schema
actually masks a problem with a tidy up that happened in !134380 (merged) because these columns have since been changed touuid
but when they were added they were varchar. But this is a problem for self-managed customers because they will get typecast errors like I was seeing likeERROR: operator does not exist: character varying = uuid # LINE 1: ...erability_occurrences WHERE vulnerability_occurrences.uuid = # ^ # HINT: No operator matches the given name and argument types. You might need to add explicit type casts. # ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
- Once I reverted the changes in !134380 (merged) I ran into yet another problem where these migrations are depending on
Vulnerabilities::CreateService
which has changed since they were created. This again might very well mean these migrations do not work for self-managed customers where they have not been run yetActiveModel::UnknownAttributeError: unknown attribute 'finding_id' for Vulnerability. raise UnknownAttributeError.new(self, k.to_s) ^^^^^ # ./gems/activerecord-gitlab/lib/active_record/gitlab_patches/rescue_from.rb:40:in `_assign_attributes' # ./ee/app/services/vulnerabilities/create_service.rb:53:in `save_vulnerability' # ./ee/app/services/vulnerabilities/create_service.rb:36:in `block in execute'
Solution
- Figure out how to fix the changes to the migration so that they can successfully run if someone upgrades via a supported upgrade path
- Finalize the migrations
- Clean up the migrations somehow so that we can add a new not-null column in !135691 (closed) without breaking the specs. Ideally this would be deleting the migration but we need to wait for a required stop upgrade point to delete them