Convert Vulnerability objects to Vulnerabilities::Read before ES queue
What does this MR do and why?
Converts Vulnerability objects to Vulnerabilities::Read records before pushing to the Elasticsearch processing queue. This prepares for the Vulnerabilities Across Contexts feature where a single vulnerability can have multiple context-specific read entries.
All changes are gated behind the convert_vulnerability_to_read_for_es_queue feature flag. When disabled, existing behavior is preserved.
Changes
New Elastic::VulnerabilitiesSearch concern (ee/app/models/concerns/elastic/vulnerabilities_search.rb)
- Replaces direct
Elastic::ApplicationVersionedSearchinclusion onVulnerability - Overrides
maintain_elasticsearch_create,maintain_elasticsearch_update, andmaintain_elasticsearch_destroyto trackvulnerability_readinstead of the vulnerability itself when the feature flag is enabled - Caches
vulnerability_readin abefore_destroycallback to avoid a race condition with DB cascade delete (vulnerability_readshasON DELETE CASCADEonvulnerability_id, so the read is gone byafter_committime)
BulkEsOperationService updates (ee/app/services/vulnerabilities/bulk_es_operation_service.rb)
- When processing Vulnerability relations, splits eligible records into convertable (flag enabled) and non-convertable, tracking reads and vulnerabilities in separate
ProcessBookkeepingService.track!calls - Preloads
vulnerability_readassociation alongside existing preloads - Shares preloaded
projectfrom Vulnerability to itsvulnerability_readviaassociation(:project).targetto avoid N+1 queries (they share the sameproject_idbut maintain separate association caches)
RemoveFromProjectService updates (ee/app/services/vulnerabilities/removal/remove_from_project_service.rb)
- Replaces
BulkEsOperationServiceusage with directProcessBookkeepingService.track!(*es_references)call - New
build_es_referencesmethod captures elastic reference strings before deletion, choosingVulnerabilities::ReadorVulnerabilityrecords based on the feature flag - Uses
association(:project).target = projectandNamespaceRootAncestorPreloaderto avoid N+1 queries
EE::Vulnerability model (ee/app/models/ee/vulnerability.rb)
- Includes
Elastic::VulnerabilitiesSearchinstead ofElastic::ApplicationVersionedSearch - Adds
convert_to_read_enabled?method - Removes
extend ::Gitlab::Utils::Override(no longer needed sinceoverride :use_elasticsearch?moved to the new concern)
Feature flag (ee/config/feature_flags/wip/convert_vulnerability_to_read_for_es_queue.yml)
- New WIP feature flag gating the conversion
Specs updated
-
vulnerabilities_search_spec.rb— Full spec for the new concern (create/update/destroy callbacks, caching, flag-disabled fallback) -
bulk_es_operation_service_spec.rb— Tests for read conversion, non-conversion for Read relations, N+1 checks, flag-disabled context, indexing-disabled context -
remove_from_project_service_spec.rb— Tests usingelastic_referenceexpectations, flag-disabled context -
vulnerability_spec.rb— Parameterized shared example withexpected_es_tracked_objectdefaulting tovulnerability.vulnerability_read, flag-disabled context
Issue: #591435
How to set up and validate locally
- Enable the feature flag:
Feature.enable(:convert_vulnerability_to_read_for_es_queue) - Create or update a vulnerability — observe that
Vulnerabilities::Readrecords are tracked in the ES bookkeeping queue instead ofVulnerabilityrecords - Disable the flag and verify the original
Vulnerabilitytracking behavior is preserved
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.