Skip default branch reindex for projects with nothing to reindex

What does this MR do and why?

This MR stops Repositories::DefaultBranchChangedEvent from queueing the two default branch Elasticsearch reindex workers for projects that have nothing to reindex. It also makes the jobs queued by this event log the project they are for.

New repositories and first pushes publish this event, so almost every new project queues ReindexVulnerabilitiesOnDefaultBranchChangeWorker and ReindexSbomOccurrencesOnDefaultBranchChangeWorker. On GitLab.com each worker ran about 250k jobs in one week. About 99% of the projects created in that week had no vulnerability_reads or sbom_occurrence_refs rows, so those jobs found nothing to do.

Each worker gets a self.dispatch?(event) method that the subscription's if: lambda calls last, after the existing Elasticsearch checks:

  • ReindexVulnerabilitiesOnDefaultBranchChangeWorker checks Vulnerabilities::Read.by_projects(project_id).exists?.
  • ReindexSbomOccurrencesOnDefaultBranchChangeWorker checks Sbom::OccurrenceRef.by_project(project_id).exists?.

These are the relations the workers walk, so the check cannot skip a project that has rows. project_settings.has_vulnerabilities is not used, because it can be false for a project that has vulnerability_reads rows, and it does not cover SBOM rows (discussion).

The change is behind the skip_default_branch_reindex_without_records flag (gitlab_com_derisk, off by default, project actor). When the flag is off, dispatch? returns true without running the query.

Notes for reviewers

  • The query runs in the publishing process. The if: lambda runs before the job is queued, often on the push path (PostReceive) or during project creation. The workers' defer_on_database_health_signal does not cover it. It is one exists? lookup on gitlab_sec, and it runs only after the flag and the Elasticsearch checks pass.
  • No early return in handle_event. each_batch already starts with a LIMIT 1 lookup on the same relation and stops when it finds no rows, so an extra exists? would cost the same.

Project context for the queued jobs

About 3% of these jobs had no meta.project or meta.root_namespace in the Sidekiq log, so they could not be counted per namespace. Kibana, 2026-10-26 09:55 to 2026-11-02 09:55 UTC: 8,304 of 250,028 ReindexVulnerabilitiesOnDefaultBranchChangeWorker jobs had no root namespace. Grouped by json.meta.caller_id, they came from:

Caller Jobs
BulkImports::FinishProjectImportWorker 5,539
ProjectTemplateExportWorker 1,652
Projects::Settings::RepositoryController#update 500
PUT /api/:version/projects/:id 388
Repositories::PostReceiveWorker 131
Other 94

Sidekiq jobs log the application context that was active when they were queued. For these jobs, that context had no project when the event was published, and HasRepository#after_repository_change_head does not set one. Project#after_repository_change_head now wraps the publish in Gitlab::ApplicationContext.with_context(project: self), so every job queued by a DefaultBranchChangedEvent subscriber logs the project and its root namespace. This is a logging change only, so it is not behind the feature flag.

Database queries

Both queries run on gitlab_sec, from the subscription's if: lambda. Plans are from Database Lab. Project 99999901 is a placeholder for a project created on 2026-11-02 with no rows, which is the common case. Project 278964 is gitlab-org/gitlab, which has many rows. Every plan is an index only scan with no heap fetches, touching 7 to 8 buffers. The timings are mostly cold reads on the clone.

vulnerability_reads (Vulnerabilities::Read.by_projects(project_id).exists?)

SELECT 1 AS one FROM "vulnerability_reads" WHERE "vulnerability_reads"."project_id" = 99999901 LIMIT 1
Project with no rows: 7 buffers, 3.9 ms
Limit  (cost=0.57..0.65 rows=1 width=4) (actual time=3.886..3.887 rows=0 loops=1)
   Buffers: shared hit=3 read=4
   I/O Timings: read=3.806 write=0.000
   ->  Index Only Scan using index_vulnerability_reads_for_filtered_removal on public.vulnerability_reads  (cost=0.57..244.89 rows=3100 width=4) (actual time=3.884..3.885 rows=0 loops=1)
         Index Cond: (vulnerability_reads.project_id = 99999901)
         Heap Fetches: 0
         Index Searches: 1
         Buffers: shared hit=3 read=4
         I/O Timings: read=3.806 write=0.000

Database Lab plan

Project with many rows: 8 buffers, 4.0 ms
Limit  (cost=0.57..0.65 rows=1 width=4) (actual time=3.982..3.984 rows=1 loops=1)
   Buffers: shared hit=4 read=4
   I/O Timings: read=3.854 write=0.000
   ->  Index Only Scan using index_vulnerability_reads_for_filtered_removal on public.vulnerability_reads  (cost=0.57..244.89 rows=3100 width=4) (actual time=3.981..3.982 rows=1 loops=1)
         Index Cond: (vulnerability_reads.project_id = 278964)
         Heap Fetches: 0
         Index Searches: 1
         Buffers: shared hit=4 read=4
         I/O Timings: read=3.854 write=0.000

Database Lab plan

sbom_occurrence_refs (Sbom::OccurrenceRef.by_project(project_id).exists?)

SELECT 1 AS one FROM "sbom_occurrence_refs" WHERE "sbom_occurrence_refs"."project_id" = 99999901 LIMIT 1
Project with no rows: 7 buffers, 3.4 ms
Limit  (cost=0.57..0.72 rows=1 width=4) (actual time=3.376..3.377 rows=0 loops=1)
   Buffers: shared hit=3 read=4
   I/O Timings: read=3.310 write=0.000
   ->  Index Only Scan using index_sbom_occurrence_refs_on_project_id on public.sbom_occurrence_refs  (cost=0.57..125.95 rows=817 width=4) (actual time=3.374..3.375 rows=0 loops=1)
         Index Cond: (sbom_occurrence_refs.project_id = 99999901)
         Heap Fetches: 0
         Index Searches: 1
         Buffers: shared hit=3 read=4
         I/O Timings: read=3.310 write=0.000

Database Lab plan

Project with many rows: 8 buffers, 3.0 ms
Limit  (cost=0.57..0.72 rows=1 width=4) (actual time=3.038..3.040 rows=1 loops=1)
   Buffers: shared hit=4 read=4
   I/O Timings: read=2.863 write=0.000
   ->  Index Only Scan using index_sbom_occurrence_refs_on_project_id on public.sbom_occurrence_refs  (cost=0.57..3623.47 rows=24180 width=4) (actual time=3.035..3.036 rows=1 loops=1)
         Index Cond: (sbom_occurrence_refs.project_id = 278964)
         Heap Fetches: 0
         Index Searches: 1
         Buffers: shared hit=4 read=4
         I/O Timings: read=2.863 write=0.000

Database Lab plan

References

Screenshots or screen recordings

Not applicable. This is a backend change with no UI.

How to set up and validate locally

  1. In the Rails console, find a project with vulnerability reads, one with SBOM occurrence refs, and one with neither:

    with_reads = Vulnerabilities::Read.distinct.pluck(:project_id)
    with_refs = Sbom::OccurrenceRef.distinct.pluck(:project_id)
    empty_id = Project.where.not(id: with_reads + with_refs).pick(:id)
  2. Enable the flag and check dispatch? for both workers:

    Feature.enable(:skip_default_branch_reindex_without_records)
    event = ->(id) { Repositories::DefaultBranchChangedEvent.new(data: { container_id: id, container_type: 'Project' }) }
    vuln_worker = Security::ProjectTrackedContexts::ReindexVulnerabilitiesOnDefaultBranchChangeWorker
    sbom_worker = Security::ProjectTrackedContexts::ReindexSbomOccurrencesOnDefaultBranchChangeWorker
    
    vuln_worker.dispatch?(event.(with_reads.first)) # => true
    sbom_worker.dispatch?(event.(with_refs.first))  # => true
    vuln_worker.dispatch?(event.(empty_id))         # => false
    sbom_worker.dispatch?(event.(empty_id))         # => false
  3. Disable the flag. dispatch? returns true without querying:

    Feature.disable(:skip_default_branch_reindex_without_records)
    vuln_worker.dispatch?(event.(empty_id)) # => true
    sbom_worker.dispatch?(event.(empty_id)) # => true

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

🤖 Generated with Claude Code

Edited by Veeramanikandan S

Merge request reports

Loading
Loading