Remove custom Apdex from CI deleted objects processing

Summary

This change improves how the system tracks and monitors the deletion of CI (Continuous Integration) objects.

Previously, the system used a simple success/failure metric based on whether objects were deleted within 12 hours of creation. Now it has been replaced with more detailed tracking that records how long objects existed before deletion using time buckets (0-1 hour, 1-2 hours, etc.) and whether the deletion was delayed beyond the 12-hour threshold.

The new approach provides better visibility into deletion patterns by categorizing objects into age groups and explicitly marking delayed deletions. This gives engineers more granular data to understand system performance and identify potential issues with the cleanup process.

The old "Apdex" metric (which measured user satisfaction based on response times) has been removed since it wasn't the right fit for this background cleanup process. Instead, the system now uses a simple counter that tracks total deletions with labels indicating the object's age and delay status.

It was also giving false positive readings, lowering our error budget: https://gitlab.slack.com/archives/C05B0MER7LM/p1770750242550869?thread_ts=1770596155.443989&cid=C05B0MER7LM

How To Test Locally

  1. You can test this in the console by mocking the items (if you don't have deletedObjects)
# Reset memoized counter
Gitlab::Metrics::CiDeletedObjectProcessingSlis.instance_variable_set(:@deletions_counter, nil)

# Create a mock object and track it
object = Struct.new(:created_at).new(6.hours.ago)
Gitlab::Metrics::CiDeletedObjectProcessingSlis.track_deletion(object)

# Now check the counter
counter = Prometheus::Client.registry.get(:ci_deleted_objects_total)
counter.values
# => {{:delayed=>"false", :age_bucket=>"4-8h"}=>1.0}

# Track another with different age
object2 = Struct.new(:created_at).new(17.hours.ago)
Gitlab::Metrics::CiDeletedObjectProcessingSlis.track_deletion(object2)

counter.values
# => {{:delayed=>"false", :age_bucket=>"4-8h"}=>1.0, {:delayed=>"true", :age_bucket=>"12-24h"}=>1.0}

Or to see the actual export metrics

  1. In GDK.yml - enable exporter: https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/gdk.example.yml#L185
    1. Gdk reconfigure & Gdk restart
  2. Then hitting the curl endpoint you should see
maxfan@mfan--20251119-K62HL:~/workspace/gdk/gitlab (duo-edit-20260210-190244) $ curl -s http://localhost:3807/metrics | grep "deleted"
# HELP ci_deleted_objects_total Multiprocess metric
# TYPE ci_deleted_objects_total counter
ci_deleted_objects_total{age_bucket="0-1h",delayed="false"} 8
# HELP gitlab_sli_ci_deleted_objects_processing_error_total Multiprocess metric
# TYPE gitlab_sli_ci_deleted_objects_processing_error_total counter
gitlab_sli_ci_deleted_objects_processing_error_total{feature_category="continuous_integration"} 0
# HELP gitlab_sli_ci_deleted_objects_processing_total Multiprocess metric
# TYPE gitlab_sli_ci_deleted_objects_processing_total counter
gitlab_sli_ci_deleted_objects_processing_total{feature_category="continuous_integration"} 8
Edited by Max Fan

Merge request reports

Loading