Skip to content

Add background migrations for Elasticsearch

Dmitry Gruzd requested to merge 234046-es-index-migrations into master

What does this MR do?

This MR implements elastic migration framework, which looks and behaves similar to rails database migrations.

Migrations are stored in ee/elastic/migrate/ with YYYYMMDDHHMMSS_migration_name.rb file name for MigrationName class.

# frozen_string_literal: true

class MigrationName < Elastic::Migration
  def migrate
  end
end

Applied migrations are stored in gitlab-#{Rails.env}-migrations index. All unexecuted migrations are applied by Elastic::MigrationWorker cron worker sequentially.

New index example
❯ curl -s -H 'Content-type: application/json' -XPOST 'http://localhost:9200/gitlab-development-migrations/_search' -d '{}' | jq
{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "gitlab-development-migrations",
        "_type": "_doc",
        "_id": "20201105181100",
        "_score": 1,
        "_source": {
          "completed": true
        }
      }
    ]
  }
}

#234046 (closed)

Screenshots (strongly suggested)

Useful links

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team

Related to #234046 (closed)

Edited by Dmitry Gruzd

Merge request reports