BBM - Add estimated time remaining
Ref: #583506 (closed)
What does this MR do and why?
This MR adds estimated_time_remaining field to the batched background migrations API.
It will now display a human-readable estimate of how long a migration will take to complete.
How it works
The estimation is calculated based on:
- Progress percentage (migrated tuples / total tuples)
- Total duration of all succeeded jobs
- Add sleeping intervals between each job
Formula: remaining = remaining_batches * max(average_job_time, interval)
Testing
- Create a new BBM
- Fetch the data from the API
Click to expand
Gitlab::Database::SharedModel.using_connection(ActiveRecord::Base.connection) do
migration = Gitlab::Database::BackgroundMigration::BatchedMigration.create!(
job_class_name: 'TestMigration',
table_name: 'users',
column_name: 'id',
interval: 120,
min_value: 1,
max_value: 1000,
batch_size: 100,
sub_batch_size: 10,
total_tuple_count: 100,
status: 1,
gitlab_schema: :gitlab_main
)
migration.batched_jobs.create!(
status: 3,
batch_size: 50,
sub_batch_size: 10,
pause_ms: 100,
min_value: 1,
max_value: 50,
started_at: 60.seconds.ago,
finished_at: Time.current
)
puts "Migration ID: #{migration.id}"
end
➜ curl --header "PRIVATE-TOKEN: <API_TOKEN>" "http://gdk.test:3000/api/v4/admin/batched_background_migrations" | jq '.[] | select(.id == <BBM_ID>)'
{
"id": 25,
"job_class_name": "TestMigration",
"table_name": "users",
"column_name": "id",
"status": "active",
"progress": 50,
"created_at": "2026-01-29T14:02:22.561Z",
"estimated_time_remaining": "1 minute"
}
Database
Single efficient query using PostgreSQL's EXTRACT(EPOCH FROM ...) to sum job durations server-side.
https://console.postgres.ai/gitlab/gitlab-production-ci/sessions/47806/commands/144321
Edited by Max Orefice