Skip to content

Add prometheus metrics for batched migrations

Andreas Brandl requested to merge ab/bm-prom-metrics into master

What does this MR do?

This introduces a couple of prometheus metrics to track the progress and internal statistics for batched migrations. This will allow us to tune parameters for the migrations better, especially on GitLab.com.

The use case currently is the primary key migrations we're running.

The batched migration feature is behind a feature flag and not yet released. Hence, we might not want to describe the metrics in documentation. I already had that, so there's that for reference:

Data migration metrics

The progress of data migrations can be observed through the following metrics:

Metric Type Since Description Labels
batched_migration_job_batch_size Gauge 12.11 Batch size for a batched job migration_id, migration_identifier
batched_migration_job_sub_batch_size Gauge 12.11 Sub-batch size for a batched migration job migration_id, migration_identifier
batched_migration_job_updated_tuples Counter 12.11 Number of tuples updated by migration job migration_id, migration_identifier
batched_migration_job_timings Histogram 12.11 Timings for a batched migration job migration_id, migration_identifier
batched_migration_job_time_efficiency Histogram 12.11 Ratio of job duration to interval migration_id, migration_identifier

Does this MR meet the acceptance criteria?

Conformity

Edited by Andreas Brandl

Merge request reports