Add index_sbom_occurrence_refs task to seed ES index on setup

What does this MR do and why?

This MR adds an index_sbom_occurrence_refs task to Search::RakeTaskExecutorService, adds :sbom_occurrence_refs to Search::Elastic::TriggerIndexingWorker::TASKS so initial setup seeds it, and exposes the rake task gitlab:elastic:index_sbom_occurrence_refs. The task is gated on Search::Elastic::SbomOccurrenceRefIndexHelper.indexing_allowed? and logs a skip message when indexing is off.

It reuses Sbom::BulkEsOperationService rather than calling track! directly, since Sbom::OccurrenceRef#es_parent resolves project.namespace.root_ancestor and that service already owns the required preloading. That service gains an optional bookkeeping_service: keyword so initial setup enqueues onto Elastic::ProcessInitialBookkeepingService instead of the incremental queue; existing callers keep the default.

References

  1. Relates to - #612844 (closed)
  2. Reference MR - !197147 (merged)

How to set up and validate locally

  1. Empty the index:

    curl -XPOST "http://localhost:9200/gitlab-development-sbom_occurrence_refs/_delete_by_query?refresh=true" -H "Content-Type: application/json" -d '{"query":{"match_all":{}}}'
    curl -s "http://localhost:9200/gitlab-development-sbom_occurrence_refs/_count"
  2. In bundle exec rails console, run the task:

    Search::RakeTaskExecutorService.new(logger: Logger.new($stdout)).execute(:index_sbom_occurrence_refs)

    Expect Indexing SBOM occurrence refs... then ... done, and Elastic::ProcessInitialBookkeepingService.queue_size to equal Sbom::OccurrenceRef.count.

  3. Drain the queue, repeating until it returns [0, 0]. The second number is the failure count and must stay 0:

    Elastic::ProcessInitialBookkeepingService.new.execute
  4. Refresh and count. The document count should equal Sbom::OccurrenceRef.count:

    curl -XPOST "http://localhost:9200/gitlab-development-sbom_occurrence_refs/_refresh"
    curl -s "http://localhost:9200/gitlab-development-sbom_occurrence_refs/_count"

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Rushik Subba

Merge request reports

Loading
Loading