Skip to content

Recreate async index `index_vuln_findings_on_uuid_including_vuln_id`

What does this MR do and why?

This migration was originally added for issue 390198. The migration ended up failing due to white-space being included in the SQL definition.

To prevent this going forward:

  1. a before_validation callback was added to the async model class to strip whitespace
  2. definition.to_s.strip was added to the migration helper as suggested by this comment

to correct the existing records in production

  1. a migration was added that updates the existing records by calling strip on the definition column

Changelog: fixed

How to set up and validate locally

  1. before running the migration, you can create an invalid record in the database with the following lambda:
    lambda {
      index_name = 'index_vuln_findings_on_uuid_including_vuln_id'
      index_sql = <<-SQL
        CREATE UNIQUE INDEX CONCURRENTLY #{index_name} ON vulnerability_occurrences (uuid) include (vulnerability_id);
      SQL
    
      async_index = Gitlab::Database::AsyncIndexes::PostgresAsyncIndex.find_or_create_by!(name: index_name) do |rec|
        rec.table_name = :vulnerability_occurrences
        rec.definition = index_sql
      end
    
      async_index.update_column(:definition, index_sql)
    }.call
  2. check that an async index was created and that the definition field starts with whitespace
    [3] pry(main)> Gitlab::Database::AsyncIndexes::PostgresAsyncIndex.all
    
    => [#<Gitlab::Database::AsyncIndexes::PostgresAsyncIndex:0x00007fdf6021b2d0
      id: 10,
      created_at: Mon, 10 Jul 2023 23:38:47.973580000 UTC +00:00,
      updated_at: Mon, 10 Jul 2023 23:38:47.973580000 UTC +00:00,
      name: "index_vuln_findings_on_uuid_including_vuln_id",
      definition: "    CREATE UNIQUE INDEX CONCURRENTLY index_vuln_findings_on_uuid_including_vuln_id ON vulnerability_occurrences (uuid) include (vulnerability_id);\n",
      table_name: "vulnerability_occurrences",
      attempts: 0,
      last_error: nil>]
  3. run the migration: bundle exec rails db:migrate
  4. check the db again, the whitespace should have been removed:
    [1] pry(main)> Gitlab::Database::AsyncIndexes::PostgresAsyncIndex.all
    
    => [#<Gitlab::Database::AsyncIndexes::PostgresAsyncIndex:0x00007ff58c287718
      id: 10,
      created_at: Mon, 10 Jul 2023 23:38:47.973580000 UTC +00:00,
      updated_at: Mon, 10 Jul 2023 23:53:44.272120000 UTC +00:00,
      name: "index_vuln_findings_on_uuid_including_vuln_id",
      definition: "CREATE UNIQUE INDEX CONCURRENTLY index_vuln_findings_on_uuid_including_vuln_id ON vulnerability_occurrences (uuid) include (vulnerability_id);",
      table_name: "vulnerability_occurrences",
      attempts: 0,
      last_error: nil>]
    [2] pry(main)> 
    

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #413010 (closed)

Edited by Michael Becker

Merge request reports