Add migration to backfill token_status for vulnerabilities in ES
What does this MR do and why?
Add migration to backfill token_status
in vulnerabilities
bundle exec ruby scripts/elastic-migration backfill_token_status_in_vulnerabilities
References
Issue: Backfill migration for token_status field (#572848 - closed)
Mapping MR: - !203119 (merged)
Population MR: - !204042 (merged)
Screenshots or screen recordings
Before | After | |
---|---|---|
25_36 | ![]() |
![]() |
25_37 | ![]() |
![]() |
How to set up and validate locally
0. Follow the ES installation instructions first.
1. Apply temporary patch
Rollback schema version in the Vulnerability
ES reference:
diff --git a/ee/lib/search/elastic/references/vulnerability.rb b/ee/lib/search/elastic/references/vulnerability.rb
index 6703054034811f..1f6d0f61066238 100644
--- a/ee/lib/search/elastic/references/vulnerability.rb
+++ b/ee/lib/search/elastic/references/vulnerability.rb
@@ -7,7 +7,7 @@ class Vulnerability < Reference
include Search::Elastic::Concerns::DatabaseReference
include ::Gitlab::Utils::StrongMemoize
- SCHEMA_VERSION = 25_37
+ SCHEMA_VERSION = 25_36
DOC_TYPE = 'vulnerability'
INDEX_NAME = 'vulnerabilities'
2. Re-index vulnerabilities at old schema
GITLAB_SIMULATE_SAAS=1 bundle exec rails console
Vulnerabilities::Read.find_each do |v|
Elastic::ProcessBookkeepingService.track!(
Search::Elastic::References::Vulnerability.new(
v.vulnerability_id,
"group_#{v.project.namespace.root_ancestor.id}"
)
)
end
Elastic::ProcessBookkeepingService.new.execute # run until [0,0]
3. Verify schema before migration
- Docs on
2536
(expected >0):
curl -s "http://localhost:9200/gitlab-development-vulnerabilities/_count?pretty" \
-H "Content-Type: application/json" \
-d '{"query": {"term": {"schema_version": 2536}}}'
- Docs on
2537
(expected = 0):
curl -s "http://localhost:9200/gitlab-development-vulnerabilities/_count?pretty" \
-H "Content-Type: application/json" \
-d '{"query": {"term": {"schema_version": 2537}}}'
4. Restore schema and run migration
- Reset schema back to
25_37
- Restart console and jobs:
GITLAB_SIMULATE_SAAS=1 bundle exec rails console
gdk rails-background-jobs
- Run migration and bookkeeping:
Elastic::DataMigrationService[20251001130713].migrate
Elastic::ProcessInitialBookkeepingService.new.execute
5. Verify schema after migration
Repeat the same count queries as in step 3.
Expected:
- All docs are on
2537
- Count for
2536
is0
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.