BBM - Add estimated time remaining

Ref: #583506 (closed)

What does this MR do and why?

This MR adds estimated_time_remaining field to the batched background migrations API.

It will now display a human-readable estimate of how long a migration will take to complete.

How it works

The estimation is calculated based on:

  • Progress percentage (migrated tuples / total tuples)
  • Total duration of all succeeded jobs
  • Add sleeping intervals between each job
Formula: remaining = remaining_batches * max(average_job_time, interval)

Testing

  • Create a new BBM
  • Fetch the data from the API
Click to expand
Gitlab::Database::SharedModel.using_connection(ActiveRecord::Base.connection) do
  migration = Gitlab::Database::BackgroundMigration::BatchedMigration.create!(
    job_class_name: 'TestMigration',
    table_name: 'users',
    column_name: 'id',
    interval: 120,
    min_value: 1,
    max_value: 1000,
    batch_size: 100,
    sub_batch_size: 10,
    total_tuple_count: 100,
    status: 1,
    gitlab_schema: :gitlab_main
  )
  
  migration.batched_jobs.create!(
    status: 3,
    batch_size: 50,
    sub_batch_size: 10,
    pause_ms: 100,
    min_value: 1,
    max_value: 50,
    started_at: 60.seconds.ago,
    finished_at: Time.current
  )
  
  puts "Migration ID: #{migration.id}"
end
➜  curl --header "PRIVATE-TOKEN: <API_TOKEN>" "http://gdk.test:3000/api/v4/admin/batched_background_migrations" | jq '.[] | select(.id == <BBM_ID>)'
{
  "id": 25,
  "job_class_name": "TestMigration",
  "table_name": "users",
  "column_name": "id",
  "status": "active",
  "progress": 50,
  "created_at": "2026-01-29T14:02:22.561Z",
  "estimated_time_remaining": "1 minute"
}

Database

Single efficient query using PostgreSQL's EXTRACT(EPOCH FROM ...) to sum job durations server-side.

https://console.postgres.ai/gitlab/gitlab-production-ci/sessions/47806/commands/144321

Edited by Max Orefice

Merge request reports

Loading