Skip to content

Rerun ConvertStringifiedRawMetadataHashToJson migration

Simon Tomlinson requested to merge invalid-location-metadata-issue-347145 into master

In !76151 (merged) we added a background migration to clean up some invalid records in the database (fixes #347145 (closed)). Curiously, there are still some records with invalid data in production after the migration has completed. We confirmed that no new records were introduced post-migration by checking the MAX(created_at) timestamps on the invalid records. None of the invalid records were created after ~Jan 12 when the migration ran. We also tested the migration on some database lab clones using this MR. The count of records updated by those tests is equal to the number of invalid records in production. So, we believe that we can clean up the remaining invalid records by running the migration again.

It's unclear to us why the migration did not get all the records the first time that it ran. We retrieved migration logs from the archives via gitlab-com/gl-infra/scalability#1552 (comment 842277890). The only errors appearing in the logs are these:

{
  "time": "2022-01-12T03:51:32.614+00:00",
  "caller_id": "Gitlab::BackgroundMigration::FixVulnerabilityOccurrencesWithHashesAsRawMetadata",
  "class": "BackgroundMigrationWorker",
  "error_message": "uninitialized constant Gitlab::BackgroundMigration::Gitlab\nDid you mean?  GitHub",
  "message": "BackgroundMigrationWorker JID-c5c977e4e07319e591f211d5: fail: 0.050969 sec",
  "type": "sidekiq"
}

These transactions were likely rolled back due to the NameError occurring. This MR fixes the namespacing issue and also re-runs the migration.

Edited by Brian Williams

Merge request reports

Loading