Skip to content

Migration to fix wrong values of label_ids in issue index for work_items

Ravi Kumar requested to merge 416091-migration-fix-label_ids-in-work_items into master

What does this MR do and why?

When we removed the labels from work_items, there was a bug not updating the label_ids in the issues index for the work_items. This MR is to reindex all the issues of type work_items which don't have new SCHEMA_VERSION = 23_09. The schema_version with 2309 is set after the underlying bug gets fixed.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  • Ensure elasticsearch is enabled
  • For a fresh test, create a new project. Create an issue in this project. And create a subtask of this issue.
  • Update the schema_version and label_ids for this issue in the elastic by running the following command. Replace the project_id with the one you have just created.
curl -XPOST "http://localhost:9200/gitlab-development-issues/_update_by_query" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "project_id": project_id
          }
        },
        {
          "term": {
            "type": "work_item"
          }
        }
      ]
    }
  },
  "script": {
    "source": "ctx._source.schema_version = 2308;ctx._source.label_ids=[0]"
  }
}' | json_pp
  • Ensure on Elastic that the issue has been updated successfully by running the following command. Verify that the schema_version is 2308 and label_ids is [0]. Replace the project_id with the one you just created.
curl -XGET "http://localhost:9200/gitlab-development-issues/_search" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "project_id": project_id
          }
        },
        {
          "term": {
            "type": "work_item"
          }
        }
      ]
    }
  }
}' | json_pp
  • Open the rails console
bundle exec rails c
  • Run the following command
Elastic::DataMigrationService[20230908161822].send(:migration).migrate
  • And then again run the above XGET curl command and ensure the schema_version is 2309 and the label_ids is an empty array. It might take a few seconds to reflect the changes. If it takes longer than 1 minute then restart the rails-background-jobs and check again.
gdk restart rails-background-jobs

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #416091 (closed)

#424581 (closed)

Runtime

~32 minutes

[1] pry(main)> documents_count = 282239
=> 282239
[2] pry(main)> throttle_delay = 1.minute
=> 1 minute
[3] pry(main)> batch_size = 9_000
=> 9000
[4] pry(main)> ((documents_count / batch_size) * throttle_delay)
=> 31 minutes
Edited by Ravi Kumar

Merge request reports