Skip to content

Backfill namespace_ancestry for issues

What does this MR do?

This MR backfills the namespace_ancestry field for all issue documents in the Elasticsearch index. The logic for calculating the field content has been done in !68796 (merged).

Additionally, this MR extracts the logic for future backfills to Elastic::MigrationBackfillHelper and uses it for the BackfillNamespaceAncestryForIssues advanced search migration.

#335825 (closed)

Migration

Note: Ensure you have Elasticsearch enabled and setup for gdk. It is important that initial Elasticsearch setup is done on the default branch and not this branch. Otherwise the migration will already be marked as "completed" and you will not be able to test it using the steps below.

  1. open rails console
  2. run the migration using the background job: Elastic::MigrationWorker.new.perform and ElasticIndexInitialBulkCronWorker.new.perform to flush the changes to Elasticsearch
  3. follow along in the log/elasticsearch.log file
    {"severity":"INFO","time":"2021-08-25T11:20:11.035Z","correlation_id":null,"message":"MigrationWorker: migration[BackfillNamespaceAncestryForIssues] executing migrate method"}
    {"severity":"INFO","time":"2021-08-25T11:20:11.043Z","correlation_id":null,"message":"[Elastic::Migration: 20210825110300] Checking if there are documents without namespace_ancestry field: 47427 documents left"}
    {"severity":"INFO","time":"2021-08-25T11:20:11.043Z","correlation_id":null,"message":"[Elastic::Migration: 20210825110300] Adding namespace_ancestry field to gitlab-development-issues documents for batch of 5000 documents"}
  4. depending on a number of documents left, you'd need to run Elastic::MigrationWorker.new.perform and ElasticIndexInitialBulkCronWorker.new.perform a few times until the migration is completed

Screenshots or Screencasts (strongly suggested)

How to setup and validate locally (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team

Related to #335825 (closed)

Edited by Dmitry Gruzd

Merge request reports

Loading