Skip to content
Snippets Groups Projects

Add schema_version to main index mapping

Merged Terri Chu requested to merge 214601-add-schema_version-to-main-index into master
All threads resolved!

What does this MR do and why?

Related to #214601 (closed)

Adding schema_version to index mapping for the main index (contains, project, wiki_blobs, and blob document types).

Note: We would like to have a schema_version field available for future migrations in all document types.

This is prep work to be able to easily backfill traversal_ids for projects which will be needed for the IndexIntegrityWorker. The IndexIntegrityWorker (dev in progress) will need to pull all projects for a root namespace and I need the traversal_ids field populated for that. Currently it's only being populated for blobs and wiki_blobs.

Screenshots or screen recordings

N/A

How to set up and validate locally

prerequisites

setup Elasticsearch in gdk

check config.rb changes

  1. create an index from scratch: bundle exec rake gitlab:elastic:index
  2. verify the new mapping is there and has data (tests the config.rb changes) GET http://localhost:9200/gitlab-development/_search
{
	"query": {
		"bool": {
			"must": [
				{
					"term": {
						"type": {
							"value": "project"
						}
					}
				}
			]
		}
	}
}

migration changes

  1. checkout the master branch
  2. recreate the indexes from scratch : bundle exec rake gitlab:elastic:index
  3. add a new project
  4. verify that it doesn't throw an error and indexes properly (without the new field)
  5. stop rails-background-jobs to prevent it from running the migration automatically: gdk stop rails-background-jobs
  6. start rails console
  7. run the migration in the rails console
  8. verify that the schema_version field exists in the mappings: GET http://localhost:9200/gitlab-development/_mappings
  9. add a new or update an existing project and run the indexing manually: Elastic::ProcessBookkeepingService.new.execute
  10. verify that the schema_version field exists and is populated for the project in the search response GET http://localhost:9200/gitlab-development/_search
{
	"query": {
		"bool": {
			"must": [
				{
					"term": {
						"type": {
							"value": "project"
						}
					}
				}
			]
		}
	}
}

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Terri Chu

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Ravi Kumar
  • Ravi Kumar requested review from @dgruzd

    requested review from @dgruzd

  • Dmitry Gruzd approved this merge request

    approved this merge request

  • :wave: @dgruzd, thanks for approving this merge request.

    This is the first time the merge request is approved. To ensure full test coverage, a new pipeline will be started shortly.

    For more info, please refer to the following links:

  • Dmitry Gruzd resolved all threads

    resolved all threads

  • Dmitry Gruzd enabled an automatic merge when the pipeline for df7b00f0 succeeds

    enabled an automatic merge when the pipeline for df7b00f0 succeeds

  • Ravi Kumar approved this merge request

    approved this merge request

  • merged

  • Dmitry Gruzd mentioned in commit 566882df

    mentioned in commit 566882df

  • added workflowstaging label and removed workflowcanary label

  • mentioned in issue #384597 (closed)

  • Please register or sign in to reply
    Loading