Remove backfill_ci_finished_builds BBM and mode-split logic

What does this MR do and why?

Cleans up the BackfillCiFinishedBuildsToClickHouse batched background migration (BBM) and all related backfill-specific logic. The BBM was finalized in 19.0; per the BBM cleanup guidelines, the migration code and its tests can be removed after the next required stop.

Closes #592894.

Note on sequencing: the previous attempt at this cleanup (!241625 (closed)) was closed unmerged, so none of this landed in 19.2. The sibling index-removal MR !241619 (merged) did merge in 19.2, so the temporary mode-filter index is already gone from production. That makes removing the mode-split query path here overdue rather than cosmetic.

What this MR removes

1. BBM code and migrations (full manual cleanup)

Following the docs' "Delete your batched background migration code manually" procedure, the whole BBM set is removed:

  • lib/gitlab/background_migration/backfill_ci_finished_builds_to_click_house.rb
  • ee/lib/ee/gitlab/background_migration/backfill_ci_finished_builds_to_click_house.rb (+ EE spec)
  • db/docs/batched_background_migrations/backfill_ci_finished_builds_to_click_house.yml
  • db/post_migrate/20260204083514_queue_...rb and db/post_migrate/20260413072219_finalize_...rb
  • The matching db/schema_migrations/20260204083514 and db/schema_migrations/20260413072219 entries
  • spec/migrations/20260204083514_queue_..._spec.rb

The queue and finalize migrations are deleted together with the code and dictionary so nothing is left orphaned. Both versions predate the last required stop and are not present in structure.sql, so there is no schema drift.

2. CiFinishedBuildsConsistencyHelper and all backfill_in_progress? usage

  • lib/click_house/migration_support/ci_finished_builds_consistency_helper.rb (+ spec)
  • backfill_in_progress? branch in ci_finished_builds_sync_cron_worker.rb
  • use_deduplicated_finder? branch in lib/ci/job_analytics/query_builder.rb. The query builder now always uses FinishedBuildsFinder (or the siphon finder when job_analytics_siphon is enabled). The redundant returns_final_query? wrapper is inlined to a direct siphon_finder? check.
  • Backfill stubs in the query builder and GraphQL job analytics specs

3. Backfill mode-split logic (from !226068 (merged))

  • recent/backfill worker split in ci_finished_builds_sync_cron_worker.rb
  • mode parameter, MODES, RECENT_THRESHOLD, and apply_mode_filter in ci_finished_builds_sync_service.rb
  • order_by_build_finished_at scope in finished_build_ch_sync_event.rb
  • The temporary GitLab.com five-worker special case in the cron worker, which only existed to drain the backfill-era sync-event backlog. The cron now uses the argument-driven worker count.
  • The dead loggable_arguments declarations on both workers (all remaining args are integers, which are never logged via that mechanism).

Deferred to the follow-up (#604160)

  • CiFinishedBuildsSyncWorker#perform keeps a deprecated _mode = nil argument (ignored) for one release, per the Sidekiq compatibility guidelines, so any in-flight jobs enqueued by the old cron path still execute.
  • ClickHouse::Finders::Ci::FinishedBuildsDeduplicatedFinder is now unused in production but is removed in the follow-up to keep this MR focused.

Feature flag

ci_finished_builds_ch_sync_mode_split was already removed from master in !227233 (merged), so there is nothing to do here.

Database

This MR only deletes an already-finalized BBM and its applied migration records. There are no new or reversible migrations to run. Deleting the schema_migrations rows matches the documented manual-cleanup steps and does not change structure.sql.

How to set up and validate locally

The change is a removal, so validation is mainly that the sync path still works without the backfill/mode-split machinery.

$ bundle exec rspec \
    ee/spec/models/ci/finished_build_ch_sync_event_spec.rb \
    ee/spec/services/click_house/data_ingestion/ci_finished_builds_sync_service_spec.rb \
    ee/spec/workers/click_house/ci_finished_builds_sync_cron_worker_spec.rb \
    ee/spec/workers/click_house/ci_finished_builds_sync_worker_spec.rb \
    spec/lib/ci/job_analytics/query_builder_spec.rb \
    spec/requests/api/graphql/project/job_analytics_spec.rb

All specs pass and RuboCop is clean on the touched files.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Narendran

Merge request reports

Loading
Loading