Delete ML records in batches when a project is deleted

What does this MR do and why?

Deleting a project now removes its rows in the ten ml_* tables in bounded statements (5,000 rows for the candidate metric, param and metadata tables, 1,000 rows for the rest) before the project row is destroyed. Today project.destroy! deletes them through the ON DELETE CASCADE foreign keys in one statement inside the project deletion transaction, so the statement grows with the project's runs and metric history and a large project cannot finish within the statement timeout. The batched path is behind the batch_delete_ml_records_on_project_destroy flag (default off); with the flag off the cascade runs as before.

Design notes

Why the service and not an association. Project declares no Ml:: association, so destroy_dependent_associations_in_batches never sees these tables and the only deletion was the database cascade. Projects::DestroyService#delete_ml_records (in app/services/projects/destroy_service.rb) follows the shape of delete_environments, delete_lfs_objects_projects and delete_notes in the same service: a loop that deletes where(project_id:).limit(n) until a statement deletes fewer than n rows and exists? finds no row left, then one log line with the counts per table. A project with no row in ml_experiments or ml_models returns after those two existence checks, since every other ml_* table references one of them. Like those methods, it runs in destroy_project_related_records, outside the project.destroy! transaction, so a failed attempt keeps the rows already deleted and the next attempt deletes the rest.

Order. The key order of ML_DELETE_BATCH_SIZES is the deletion order: leaf tables first (candidate metrics, params, metadata; candidates; experiment metadata; experiments; model version metadata; model versions; model metadata; models), so no statement cascades into another ml_* table. Each deleted ml_candidates row still fires three RI_FKey_cascade_del triggers (metrics, params, metadata), each ml_experiments row two, each ml_model_versions row two and each ml_models row three; because the children are already gone, each trigger is one index probe that finds nothing (the plans below show the trigger time).

Why where(project_id:).limit(n) and not each_batch. Every ml_* table has an index that starts with project_id, so the first statement reads the first 1,000 live index entries without a sort. Later statements walk past the entries of the rows already deleted: statement k skips about (k - 1) × 1,000 dead index entries (until vacuum or the index's own dead-entry hints remove them) before it finds 1,000 live ids. The loop terminates because LIMIT counts live rows only, and the statement cost is bounded by the number of dead entries, not by the table size. delete_notes and delete_environments in the same service have the same shape. each_batch orders by id, which these indexes do not provide, so each batch would sort every remaining row of the project. ml_candidates could walk (project_id, internal_id) by keyset, but the metric, param and metadata tables have no such index, so the simple loop is used throughout. The measurements below show the late-batch cost.

Why not reuse Ml::DeleteExperimentRunsService. That service batches per experiment on eid and covers only runs and their children, which is strictly more work than one project-scoped index range per table, and it does not cover models, versions and metadata.

Batch size. ML_DELETE_BATCH_SIZES gives 5,000 rows per statement to the three leaf tables (ml_candidate_metrics, ml_candidate_params, ml_candidate_metadata): no foreign key points at them, so a deleted row fires no referential integrity trigger, and a 5,000-row statement measured 190 to 530 ms cold and about 15 ms warm. ml_candidates and the parent tables keep 1,000 rows per statement because each deleted row runs two or three referential integrity checks (a 1,000-row ml_candidates statement is about 360 ms cold and 80 ms warm, mostly trigger time). Ml::DeleteExperimentRunsService (!264622 (merged)) uses 10,000 for metrics under a runtime limiter; project deletion runs in one Sidekiq job without one.

Expected job time for the largest projects. Every statement stays under 1 s and far under the 15 s statement timeout; the job time is the sum. The project with the most metrics on GitLab.com (10.3 M rows) needs about 2,055 statements on ml_candidate_metrics: about 31 s warm, up to 6.5 min cold. The project with the most candidates and candidate metadata (911 K candidates, 3.5 M metadata rows) needs about 709 statements on ml_candidate_metadata (10 s warm, up to 2.4 min cold), 183 on ml_candidate_metrics (3 s warm, 35 s cold) and 912 on ml_candidates (71 s warm, up to 5.4 min cold): about 1.4 min warm and up to 8.4 min cold in one ProjectDestroyWorker job. The rollout issue (https://gitlab.com/gitlab-org/gitlab/-/issues/638631) watches the job duration.

Interplay with the experiment deletion worker. Ml::DestroyExperimentWorker (!264622 (merged)) may be deleting an experiment of the same project at the same time. While both run, the second DELETE waits on the row locks of the first, and two deletes that lock overlapping rows in different orders can deadlock. The worker rescues the error and leaves its claim, so it retries the experiment after 10 minutes. The destroy service rescues the error in execute, attempt_rollback resets pending_delete and reschedule_deletion! restores the state the project had before the attempt, and execute returns false, so ProjectDestroyWorker finishes without a Sidekiq retry (retry: 3 covers raised errors only). A deletion_scheduled project is picked up again by AdjournedProjectsDeletionCronWorker; a project deleted immediately, archived or not, needs a new deletion request. Rows already deleted stay deleted, so the next attempt has less to do. A batch that comes back short because the worker deleted rows the LIMIT subquery had selected does not end the table's loop: the loop stops only when exists? finds no row of the project.

Sibling MR. !263939 (merged) (open) changes app/models/ml/* and the ML destroy services for experiment and model deletion. It does not touch app/services/projects/destroy_service.rb or its spec, so the two MRs do not conflict and can merge in any order.

Query budget of DeleteUserWorker. A hard delete of a user who solo-owns a group runs Projects::DestroyService synchronously inside DeleteUserWorker. For a project without ML rows delete_ml_records adds two existence checks, so the admin hard-delete example in spec/controllers/admin/users_controller_spec.rb runs 200 queries, under the worker's existing 205 budget; app/workers/delete_user_worker.rb is unchanged.

Rollback. Turn the flag off; the next deletion attempt uses the cascade again. The flag can be removed once the rollout is stable (https://gitlab.com/gitlab-org/gitlab/-/issues/638631).

References

Screenshots or screen recordings

No UI change.

How to set up and validate locally

  1. In the rails console, enable the flag and create a project with ML records:

    Feature.enable(:batch_delete_ml_records_on_project_destroy)
    user = User.find_by_username('root')
    project = Projects::CreateService.new(user, name: 'ml-delete-me', path: 'ml-delete-me', namespace_id: user.namespace_id).execute
    exp = Ml::Experiment.create!(project: project, user: user, name: 'exp')
    3.times { |i| c = Ml::Candidate.create!(experiment: exp, project: project, user: user, name: "run#{i}"); c.metrics.create!(name: 'loss', value: i, step: 0, tracked_at: 0); c.params.create!(name: 'lr', value: '0.1') }
    model = Ml::CreateModelService.new(project, 'model', user).execute.payload[:model]
    Ml::CreateModelVersionService.new(model, user: user, version: '1.0.0').execute
  2. Delete the project: Projects::DestroyService.new(project, user).execute. Expect: true.

  3. In the console, run %w[ml_candidate_metrics ml_candidate_params ml_candidates ml_experiments ml_model_versions ml_models].to_h { |t| [t, ApplicationRecord.connection.select_value("SELECT count(*) FROM #{t} WHERE project_id = #{project.id}")] }. Expect: every count is 0.

  4. Run grep 'Deleting ML records completed' log/application_json.log | tail -1. Expect: one line with deleted_ml_record_counts listing the ten tables, for example "ml_candidate_metrics":3,"ml_candidates":4,"ml_experiments":2,"ml_models":1.

Flag-off check

Repeat the steps with Feature.disable(:batch_delete_ml_records_on_project_destroy). Expect: the counts are also 0 (the cascade deletes the rows) and no new Deleting ML records completed line appears.

Database review: table sizes, measurements and query plans

No migration. The new statements are the ten DELETEs below, one per table, repeated until a statement deletes fewer rows than its batch size (5,000 for the three leaf tables, 1,000 for the rest). Rails generates the form DELETE FROM "t" WHERE "t"."id" IN (SELECT "t"."id" FROM "t" WHERE "t"."project_id" = $1 LIMIT $2). All plans are EXPLAIN (ANALYZE, BUFFERS) on Database Lab (gitlab-production-main, 2026-11-05), executed on the thin clone and rolled back, on the densest key of each table: ml_candidate_metrics project 51848901 (10,271,442 rows), ml_candidate_metadata and ml_candidates project 72033045 (3,540,718 and 911,166 rows), ml_candidate_params project 42951908 (952,317 rows), ml_models project 74016851 (1,165 rows). 168 projects have more than 1,000 metric rows, 99 more than 1,000 candidate metadata rows and 28 more than 1,000 candidates, of 1,809 projects with candidates. For the parent tables the children of the project were deleted in the clone first, as the service does.

Table sizes on GitLab.com (total relation size, estimated rows):

Table Size Rows
ml_candidate_metrics 7.7 GB 48.3 M
ml_candidate_metadata 1.3 GB 6.2 M
ml_candidate_params 582 MB 2.7 M
ml_candidates 442 MB 1.2 M
ml_experiments 12 MB 31.8 K
ml_model_versions 11 MB 21.8 K
ml_models 6.8 MB 8.7 K
ml_model_version_metadata 3.5 MB 17.5 K
ml_experiment_metadata 1.1 MB 4.5 K
ml_model_metadata 128 KB 102

Per statement, cold (first run after joe reset) then warm:

Table Rows per statement Key Cold Warm Buffers (cold) WAL cold / warm
ml_candidate_metrics 5,000 51848901 191 ms 15 ms hit 30,232 read 319 2.46 MB / 270 KB
ml_candidate_metrics 1,000 51848901 107 ms 3.1 ms hit 6,083 read 137 918 KB / 54 KB
ml_candidate_params 5,000 42951908 529 ms 15.5 ms hit 25,601 read 835 6.07 MB / 270 KB
ml_candidate_params 1,000 42951908 153 ms 5.7 ms hit 5,152 read 225 1.56 MB / 54 KB
ml_candidate_metadata 5,000 72033045 206 ms 14 ms hit 25,244 read 361 2.69 MB / 270 KB
ml_candidate_metadata 1,000 72033045 86 ms 6.3 ms hit 5,058 read 152 885 KB / 885 KB
ml_candidates (children gone) 1,000 56719366 62 ms + 294 ms RI triggers 3.8 ms + 74 ms RI triggers hit 6,783 read 229 54 KB / 54 KB
ml_experiments (95 rows, children gone) 1,000 56719366 7 ms + 11 ms RI triggers - hit 497 read 15 5 KB

WAL per row is 54 bytes warm on every table; the first touch of a heap page adds a full-page image (about 900 bytes per row on metrics and metadata, 1.5 KB on params). With the children still present, the same 1,000-row ml_candidates statement on project 72033045 costs 24 ms plus 2.74 s of cascade triggers and 7.9 MB of WAL, which is why the leaf tables go first.

Late batches (dead index entries): after 200,000 ml_candidate_metadata rows of project 72033045 were deleted and committed, the next 1,000-row statement's index scan read 247 buffers instead of 108 and ran in 30 ms. After 500,000 ml_candidate_metrics rows of the same project were deleted in the clone, the first following 1,000-row statement walked the dead entries (24,784 index buffer hits, 249 ms) and the next statements took 11 to 15 ms with about 650 index buffer hits, because the first scan marked the dead entries in the index (kill_prior_tuple); the plans for both are at the end.

ml_candidate_metrics (5,000 rows, project 51848901, 10,271,442 rows)
DELETE FROM "ml_candidate_metrics" WHERE "ml_candidate_metrics"."id" IN (SELECT "ml_candidate_metrics"."id" FROM "ml_candidate_metrics" WHERE "ml_candidate_metrics"."project_id" = 51848901 LIMIT 5000)

cold:

 Delete on public.ml_candidate_metrics  (cost=619.70..18430.15 rows=0 width=0) (actual time=190.631..190.634 rows=0 loops=1)
   Buffers: shared hit=30232 read=319 dirtied=270
   WAL: records=5000 fpi=269 bytes=2462045
   I/O Timings: read=170.059 write=0.000
   ->  Nested Loop  (cost=619.70..18430.15 rows=5000 width=38) (actual time=154.471..186.563 rows=5000 loops=1)
         Buffers: shared hit=24964 read=318
         I/O Timings: read=170.024 write=0.000
         ->  HashAggregate  (cost=619.13..669.13 rows=5000 width=40) (actual time=153.130..154.319 rows=5000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 721kB
               Buffers: shared hit=5 read=277
               I/O Timings: read=148.141 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.56..606.63 rows=5000 width=40) (actual time=2.059..151.286 rows=5000 loops=1)
                     Buffers: shared hit=5 read=277
                     I/O Timings: read=148.141 write=0.000
                     ->  Limit  (cost=0.56..556.63 rows=5000 width=8) (actual time=2.048..150.297 rows=5000 loops=1)
                           Buffers: shared hit=5 read=277
                           I/O Timings: read=148.141 write=0.000
                           ->  Index Scan using index_ml_candidate_metrics_on_project_id on public.ml_candidate_metrics ml_candidate_metrics_1  (cost=0.56..1133649.96 rows=10193440 width=8) (actual time=2.047..149.866 rows=5000 loops=1)
                                 Index Cond: (ml_candidate_metrics_1.project_id = 51848901)
                                 Buffers: shared hit=5 read=277
                                 I/O Timings: read=148.141 write=0.000
         ->  Index Scan using ml_candidate_metrics_pkey on public.ml_candidate_metrics  (cost=0.56..3.55 rows=1 width=14) (actual time=0.006..0.006 rows=1 loops=5000)
               Index Cond: (ml_candidate_metrics.id = "ANY_subquery".id)
               Buffers: shared hit=24959 read=41
               I/O Timings: read=21.884 write=0.000

warm:

 Delete on public.ml_candidate_metrics  (cost=619.70..18430.15 rows=0 width=0) (actual time=14.768..14.770 rows=0 loops=1)
   Buffers: shared hit=30282
   WAL: records=5000 fpi=0 bytes=270000
   I/O Timings: read=0.000 write=0.000
   ->  Nested Loop  (cost=619.70..18430.15 rows=5000 width=38) (actual time=4.284..12.782 rows=5000 loops=1)
         Buffers: shared hit=25282
         I/O Timings: read=0.000 write=0.000
         ->  HashAggregate  (cost=619.13..669.13 rows=5000 width=40) (actual time=4.264..4.997 rows=5000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 721kB
               Buffers: shared hit=282
               I/O Timings: read=0.000 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.56..606.63 rows=5000 width=40) (actual time=0.085..3.174 rows=5000 loops=1)
                     Buffers: shared hit=282
                     I/O Timings: read=0.000 write=0.000
                     ->  Limit  (cost=0.56..556.63 rows=5000 width=8) (actual time=0.076..2.387 rows=5000 loops=1)
                           Buffers: shared hit=282
                           I/O Timings: read=0.000 write=0.000
                           ->  Index Scan using index_ml_candidate_metrics_on_project_id on public.ml_candidate_metrics ml_candidate_metrics_1  (cost=0.56..1133649.96 rows=10193440 width=8) (actual time=0.075..1.976 rows=5000 loops=1)
                                 Index Cond: (ml_candidate_metrics_1.project_id = 51848901)
                                 Buffers: shared hit=282
                                 I/O Timings: read=0.000 write=0.000
         ->  Index Scan using ml_candidate_metrics_pkey on public.ml_candidate_metrics  (cost=0.56..3.55 rows=1 width=14) (actual time=0.001..0.001 rows=1 loops=5000)
               Index Cond: (ml_candidate_metrics.id = "ANY_subquery".id)
               Buffers: shared hit=25000
               I/O Timings: read=0.000 write=0.000
ml_candidate_params (5,000 rows, project 42951908, 952,317 rows)
DELETE FROM "ml_candidate_params" WHERE "ml_candidate_params"."id" IN (SELECT "ml_candidate_params"."id" FROM "ml_candidate_params" WHERE "ml_candidate_params"."project_id" = 42951908 LIMIT 5000)

cold:

 Delete on public.ml_candidate_params  (cost=450.15..15539.07 rows=0 width=0) (actual time=528.789..528.792 rows=0 loops=1)
   Buffers: shared hit=25601 read=835 dirtied=713
   WAL: records=5000 fpi=712 bytes=6074756
   I/O Timings: read=506.529 write=0.000
   ->  Nested Loop  (cost=450.15..15539.07 rows=5000 width=38) (actual time=451.794..523.076 rows=5000 loops=1)
         Buffers: shared hit=19889 read=835
         I/O Timings: read=506.529 write=0.000
         ->  HashAggregate  (cost=449.72..499.72 rows=5000 width=40) (actual time=450.354..451.427 rows=5000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 721kB
               Buffers: shared hit=3 read=721
               I/O Timings: read=443.441 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.43..437.22 rows=5000 width=40) (actual time=3.471..448.302 rows=5000 loops=1)
                     Buffers: shared hit=3 read=721
                     I/O Timings: read=443.441 write=0.000
                     ->  Limit  (cost=0.43..387.22 rows=5000 width=8) (actual time=3.459..447.140 rows=5000 loops=1)
                           Buffers: shared hit=3 read=721
                           I/O Timings: read=443.441 write=0.000
                           ->  Index Scan using index_ml_candidate_params_on_project_id on public.ml_candidate_params ml_candidate_params_1  (cost=0.43..73675.30 rows=952381 width=8) (actual time=3.458..446.681 rows=5000 loops=1)
                                 Index Cond: (ml_candidate_params_1.project_id = 42951908)
                                 Buffers: shared hit=3 read=721
                                 I/O Timings: read=443.441 write=0.000
         ->  Index Scan using ml_candidate_params_pkey on public.ml_candidate_params  (cost=0.43..3.01 rows=1 width=14) (actual time=0.014..0.014 rows=1 loops=5000)
               Index Cond: (ml_candidate_params.id = "ANY_subquery".id)
               Buffers: shared hit=19886 read=114
               I/O Timings: read=63.088 write=0.000

warm:

 Delete on public.ml_candidate_params  (cost=450.15..15539.07 rows=0 width=0) (actual time=15.548..15.550 rows=0 loops=1)
   Buffers: shared hit=25724
   WAL: records=5000 fpi=0 bytes=270000
   I/O Timings: read=0.000 write=0.000
   ->  Nested Loop  (cost=450.15..15539.07 rows=5000 width=38) (actual time=5.407..13.517 rows=5000 loops=1)
         Buffers: shared hit=20724
         I/O Timings: read=0.000 write=0.000
         ->  HashAggregate  (cost=449.72..499.72 rows=5000 width=40) (actual time=5.392..6.163 rows=5000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 721kB
               Buffers: shared hit=724
               I/O Timings: read=0.000 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.43..437.22 rows=5000 width=40) (actual time=0.070..4.350 rows=5000 loops=1)
                     Buffers: shared hit=724
                     I/O Timings: read=0.000 write=0.000
                     ->  Limit  (cost=0.43..387.22 rows=5000 width=8) (actual time=0.060..3.622 rows=5000 loops=1)
                           Buffers: shared hit=724
                           I/O Timings: read=0.000 write=0.000
                           ->  Index Scan using index_ml_candidate_params_on_project_id on public.ml_candidate_params ml_candidate_params_1  (cost=0.43..73675.30 rows=952381 width=8) (actual time=0.058..3.246 rows=5000 loops=1)
                                 Index Cond: (ml_candidate_params_1.project_id = 42951908)
                                 Buffers: shared hit=724
                                 I/O Timings: read=0.000 write=0.000
         ->  Index Scan using ml_candidate_params_pkey on public.ml_candidate_params  (cost=0.43..3.01 rows=1 width=14) (actual time=0.001..0.001 rows=1 loops=5000)
               Index Cond: (ml_candidate_params.id = "ANY_subquery".id)
               Buffers: shared hit=20000
               I/O Timings: read=0.000 write=0.000
ml_candidate_metadata (5,000 rows, project 72033045, 3,540,718 rows)
DELETE FROM "ml_candidate_metadata" WHERE "ml_candidate_metadata"."id" IN (SELECT "ml_candidate_metadata"."id" FROM "ml_candidate_metadata" WHERE "ml_candidate_metadata"."project_id" = 72033045 LIMIT 5000)

cold:

 Delete on public.ml_candidate_metadata  (cost=340.68..16492.10 rows=0 width=0) (actual time=206.203..206.206 rows=0 loops=1)
   Buffers: shared hit=25244 read=361 dirtied=298
   WAL: records=5000 fpi=297 bytes=2691401
   I/O Timings: read=188.470 write=0.000
   ->  Nested Loop  (cost=340.68..16492.10 rows=5000 width=38) (actual time=160.165..202.486 rows=5000 loops=1)
         Buffers: shared hit=19947 read=361
         I/O Timings: read=188.470 write=0.000
         ->  HashAggregate  (cost=340.24..390.24 rows=5000 width=40) (actual time=157.327..158.303 rows=5000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 721kB
               Buffers: shared hit=3 read=305
               I/O Timings: read=152.257 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.43..327.74 rows=5000 width=40) (actual time=4.978..155.613 rows=5000 loops=1)
                     Buffers: shared hit=3 read=305
                     I/O Timings: read=152.257 write=0.000
                     ->  Limit  (cost=0.43..277.74 rows=5000 width=8) (actual time=4.966..154.670 rows=5000 loops=1)
                           Buffers: shared hit=3 read=305
                           I/O Timings: read=152.257 write=0.000
                           ->  Index Scan using index_ml_candidate_metadata_on_project_id on public.ml_candidate_metadata ml_candidate_metadata_1  (cost=0.43..196522.86 rows=3543368 width=8) (actual time=4.965..154.240 rows=5000 loops=1)
                                 Index Cond: (ml_candidate_metadata_1.project_id = 72033045)
                                 Buffers: shared hit=3 read=305
                                 I/O Timings: read=152.257 write=0.000
         ->  Index Scan using ml_candidate_metadata_pkey on public.ml_candidate_metadata  (cost=0.43..3.22 rows=1 width=14) (actual time=0.009..0.009 rows=1 loops=5000)
               Index Cond: (ml_candidate_metadata.id = "ANY_subquery".id)
               Buffers: shared hit=19944 read=56
               I/O Timings: read=36.212 write=0.000

warm:

 Delete on public.ml_candidate_metadata  (cost=340.68..16492.10 rows=0 width=0) (actual time=14.044..14.047 rows=0 loops=1)
   Buffers: shared hit=25308
   WAL: records=5000 fpi=0 bytes=270000
   I/O Timings: read=0.000 write=0.000
   ->  Nested Loop  (cost=340.68..16492.10 rows=5000 width=38) (actual time=3.947..11.973 rows=5000 loops=1)
         Buffers: shared hit=20308
         I/O Timings: read=0.000 write=0.000
         ->  HashAggregate  (cost=340.24..390.24 rows=5000 width=40) (actual time=3.934..4.665 rows=5000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 721kB
               Buffers: shared hit=308
               I/O Timings: read=0.000 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.43..327.74 rows=5000 width=40) (actual time=0.059..2.980 rows=5000 loops=1)
                     Buffers: shared hit=308
                     I/O Timings: read=0.000 write=0.000
                     ->  Limit  (cost=0.43..277.74 rows=5000 width=8) (actual time=0.051..2.250 rows=5000 loops=1)
                           Buffers: shared hit=308
                           I/O Timings: read=0.000 write=0.000
                           ->  Index Scan using index_ml_candidate_metadata_on_project_id on public.ml_candidate_metadata ml_candidate_metadata_1  (cost=0.43..196522.86 rows=3543368 width=8) (actual time=0.050..1.877 rows=5000 loops=1)
                                 Index Cond: (ml_candidate_metadata_1.project_id = 72033045)
                                 Buffers: shared hit=308
                                 I/O Timings: read=0.000 write=0.000
         ->  Index Scan using ml_candidate_metadata_pkey on public.ml_candidate_metadata  (cost=0.43..3.22 rows=1 width=14) (actual time=0.001..0.001 rows=1 loops=5000)
               Index Cond: (ml_candidate_metadata.id = "ANY_subquery".id)
               Buffers: shared hit=20000
               I/O Timings: read=0.000 write=0.000
ml_candidates (1,000 rows, project 56719366, metrics, params and metadata of the project already deleted)
DELETE FROM "ml_candidates" WHERE "ml_candidates"."id" IN (SELECT "ml_candidates"."id" FROM "ml_candidates" WHERE "ml_candidates"."project_id" = 56719366 LIMIT 1000)

cold:

 Delete on public.ml_candidates  (cost=1012.35..4264.61 rows=0 width=0) (actual time=61.662..61.664 rows=0 loops=1)
   Buffers: shared hit=6783 read=229 dirtied=156
   WAL: records=1000 fpi=0 bytes=54000
   I/O Timings: read=53.796 write=0.000
   ->  Nested Loop  (cost=1012.35..4264.61 rows=1000 width=38) (actual time=10.489..57.701 rows=1000 loops=1)
         Buffers: shared hit=4777 read=225 dirtied=156
         I/O Timings: read=51.516 write=0.000
         ->  HashAggregate  (cost=1011.92..1021.92 rows=1000 width=40) (actual time=8.865..9.296 rows=1000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 193kB
               Buffers: shared hit=838 read=164 dirtied=156
               I/O Timings: read=6.381 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.43..1009.42 rows=1000 width=40) (actual time=2.451..8.534 rows=1000 loops=1)
                     Buffers: shared hit=838 read=164 dirtied=156
                     I/O Timings: read=6.381 write=0.000
                     ->  Limit  (cost=0.43..999.42 rows=1000 width=8) (actual time=2.437..8.273 rows=1000 loops=1)
                           Buffers: shared hit=838 read=164 dirtied=156
                           I/O Timings: read=6.381 write=0.000
                           ->  Index Scan using index_ml_candidates_on_project_id_and_eid on public.ml_candidates ml_candidates_1  (cost=0.43..6674.69 rows=6681 width=8) (actual time=2.435..8.161 rows=1000 loops=1)
                                 Index Cond: (ml_candidates_1.project_id = 56719366)
                                 Buffers: shared hit=838 read=164 dirtied=156
                                 I/O Timings: read=6.381 write=0.000
         ->  Index Scan using ml_candidates_pkey on public.ml_candidates  (cost=0.43..3.25 rows=1 width=14) (actual time=0.048..0.048 rows=1 loops=1000)
               Index Cond: (ml_candidates.id = "ANY_subquery".id)
               Buffers: shared hit=3939 read=61
               I/O Timings: read=45.136 write=0.000
Trigger RI_ConstraintTrigger_a_40791 for constraint fk_ml_candidate_metrics_on_candidate_id: time=222.184 calls=1000
Trigger RI_ConstraintTrigger_a_40796 for constraint fk_ml_candidate_params_on_candidate_id: time=39.549 calls=1000
Trigger RI_ConstraintTrigger_a_43380 for constraint fk_rails_5117dddf22: time=32.338 calls=1000

warm:

 Delete on public.ml_candidates  (cost=1012.35..4264.61 rows=0 width=0) (actual time=3.804..3.805 rows=0 loops=1)
   Buffers: shared hit=7012
   WAL: records=1000 fpi=0 bytes=54000
   I/O Timings: read=0.000 write=0.000
   ->  Nested Loop  (cost=1012.35..4264.61 rows=1000 width=38) (actual time=1.372..2.995 rows=1000 loops=1)
         Buffers: shared hit=5002
         I/O Timings: read=0.000 write=0.000
         ->  HashAggregate  (cost=1011.92..1021.92 rows=1000 width=40) (actual time=1.360..1.481 rows=1000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 193kB
               Buffers: shared hit=1002
               I/O Timings: read=0.000 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.43..1009.42 rows=1000 width=40) (actual time=0.041..1.182 rows=1000 loops=1)
                     Buffers: shared hit=1002
                     I/O Timings: read=0.000 write=0.000
                     ->  Limit  (cost=0.43..999.42 rows=1000 width=8) (actual time=0.035..1.025 rows=1000 loops=1)
                           Buffers: shared hit=1002
                           I/O Timings: read=0.000 write=0.000
                           ->  Index Scan using index_ml_candidates_on_project_id_and_eid on public.ml_candidates ml_candidates_1  (cost=0.43..6674.69 rows=6681 width=8) (actual time=0.034..0.938 rows=1000 loops=1)
                                 Index Cond: (ml_candidates_1.project_id = 56719366)
                                 Buffers: shared hit=1002
                                 I/O Timings: read=0.000 write=0.000
         ->  Index Scan using ml_candidates_pkey on public.ml_candidates  (cost=0.43..3.25 rows=1 width=14) (actual time=0.001..0.001 rows=1 loops=1000)
               Index Cond: (ml_candidates.id = "ANY_subquery".id)
               Buffers: shared hit=4000
               I/O Timings: read=0.000 write=0.000
Trigger RI_ConstraintTrigger_a_40791 for constraint fk_ml_candidate_metrics_on_candidate_id: time=49.286 calls=1000
Trigger RI_ConstraintTrigger_a_40796 for constraint fk_ml_candidate_params_on_candidate_id: time=12.991 calls=1000
Trigger RI_ConstraintTrigger_a_43380 for constraint fk_rails_5117dddf22: time=12.320 calls=1000
ml_experiment_metadata (1,000 rows, project 72033045)
DELETE FROM "ml_experiment_metadata" WHERE "ml_experiment_metadata"."id" IN (SELECT "ml_experiment_metadata"."id" FROM "ml_experiment_metadata" WHERE "ml_experiment_metadata"."project_id" = 72033045 LIMIT 1000)

cold:

 Delete on public.ml_experiment_metadata  (cost=3.59..6.62 rows=0 width=0) (actual time=1.628..1.630 rows=0 loops=1)
   Buffers: shared hit=3 read=2
   I/O Timings: read=1.588 write=0.000
   ->  Nested Loop  (cost=3.59..6.62 rows=1 width=38) (actual time=1.626..1.628 rows=0 loops=1)
         Buffers: shared hit=3 read=2
         I/O Timings: read=1.588 write=0.000
         ->  HashAggregate  (cost=3.31..3.32 rows=1 width=40) (actual time=1.625..1.627 rows=0 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 24kB
               Buffers: shared hit=3 read=2
               I/O Timings: read=1.588 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.28..3.31 rows=1 width=40) (actual time=1.622..1.624 rows=0 loops=1)
                     Buffers: shared hit=3 read=2
                     I/O Timings: read=1.588 write=0.000
                     ->  Limit  (cost=0.28..3.30 rows=1 width=8) (actual time=1.621..1.622 rows=0 loops=1)
                           Buffers: shared hit=3 read=2
                           I/O Timings: read=1.588 write=0.000
                           ->  Index Scan using index_ml_experiment_metadata_on_project_id on public.ml_experiment_metadata ml_experiment_metadata_1  (cost=0.28..3.30 rows=1 width=8) (actual time=1.620..1.620 rows=0 loops=1)
                                 Index Cond: (ml_experiment_metadata_1.project_id = 72033045)
                                 Buffers: shared hit=3 read=2
                                 I/O Timings: read=1.588 write=0.000
         ->  Index Scan using ml_experiment_metadata_pkey on public.ml_experiment_metadata  (cost=0.28..3.30 rows=1 width=14) (never executed)
               Index Cond: (ml_experiment_metadata.id = "ANY_subquery".id)
               I/O Timings: read=0.000 write=0.000
ml_experiments (1,000 rows, project 56719366, 95 rows, candidates and experiment metadata already deleted)
DELETE FROM "ml_experiments" WHERE "ml_experiments"."id" IN (SELECT "ml_experiments"."id" FROM "ml_experiments" WHERE "ml_experiments"."project_id" = 56719366 LIMIT 1000)

cold:

 Delete on public.ml_experiments  (cost=118.59..373.49 rows=0 width=0) (actual time=7.002..7.004 rows=0 loops=1)
   Buffers: shared hit=497 read=15 dirtied=7
   WAL: records=95 fpi=0 bytes=5130
   I/O Timings: read=6.430 write=0.000
   ->  Nested Loop  (cost=118.59..373.49 rows=91 width=38) (actual time=4.675..6.129 rows=95 loops=1)
         Buffers: shared hit=301 read=14 dirtied=7
         I/O Timings: read=5.763 write=0.000
         ->  HashAggregate  (cost=118.31..119.22 rows=91 width=40) (actual time=3.637..3.654 rows=95 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 24kB
               Buffers: shared hit=20 read=10 dirtied=7
               I/O Timings: read=3.462 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.29..118.08 rows=91 width=40) (actual time=2.458..3.613 rows=95 loops=1)
                     Buffers: shared hit=20 read=10 dirtied=7
                     I/O Timings: read=3.462 write=0.000
                     ->  Limit  (cost=0.29..117.17 rows=91 width=8) (actual time=2.446..3.582 rows=95 loops=1)
                           Buffers: shared hit=20 read=10 dirtied=7
                           I/O Timings: read=3.462 write=0.000
                           ->  Index Scan using index_ml_experiments_on_project_id_and_iid on public.ml_experiments ml_experiments_1  (cost=0.29..117.17 rows=91 width=8) (actual time=2.445..3.572 rows=95 loops=1)
                                 Index Cond: (ml_experiments_1.project_id = 56719366)
                                 Buffers: shared hit=20 read=10 dirtied=7
                                 I/O Timings: read=3.462 write=0.000
         ->  Index Scan using ml_experiments_pkey on public.ml_experiments  (cost=0.29..2.79 rows=1 width=14) (actual time=0.026..0.026 rows=1 loops=95)
               Index Cond: (ml_experiments.id = "ANY_subquery".id)
               Buffers: shared hit=281 read=4
               I/O Timings: read=2.301 write=0.000
Trigger RI_ConstraintTrigger_a_37415 for constraint fk_56d6ed4d3d: time=6.426 calls=95
Trigger RI_ConstraintTrigger_a_44359 for constraint fk_rails_6b39844d44: time=4.770 calls=95
ml_model_version_metadata (1,000 rows, project 72033045)
DELETE FROM "ml_model_version_metadata" WHERE "ml_model_version_metadata"."id" IN (SELECT "ml_model_version_metadata"."id" FROM "ml_model_version_metadata" WHERE "ml_model_version_metadata"."project_id" = 72033045 LIMIT 1000)

cold:

 Delete on public.ml_model_version_metadata  (cost=3.60..6.63 rows=0 width=0) (actual time=1.552..1.554 rows=0 loops=1)
   Buffers: shared hit=3 read=2
   I/O Timings: read=1.514 write=0.000
   ->  Nested Loop  (cost=3.60..6.63 rows=1 width=38) (actual time=1.551..1.553 rows=0 loops=1)
         Buffers: shared hit=3 read=2
         I/O Timings: read=1.514 write=0.000
         ->  HashAggregate  (cost=3.32..3.33 rows=1 width=40) (actual time=1.550..1.552 rows=0 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 24kB
               Buffers: shared hit=3 read=2
               I/O Timings: read=1.514 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.29..3.31 rows=1 width=40) (actual time=1.548..1.549 rows=0 loops=1)
                     Buffers: shared hit=3 read=2
                     I/O Timings: read=1.514 write=0.000
                     ->  Limit  (cost=0.29..3.31 rows=1 width=8) (actual time=1.548..1.548 rows=0 loops=1)
                           Buffers: shared hit=3 read=2
                           I/O Timings: read=1.514 write=0.000
                           ->  Index Scan using index_ml_model_version_metadata_on_project_id on public.ml_model_version_metadata ml_model_version_metadata_1  (cost=0.29..3.31 rows=1 width=8) (actual time=1.546..1.547 rows=0 loops=1)
                                 Index Cond: (ml_model_version_metadata_1.project_id = 72033045)
                                 Buffers: shared hit=3 read=2
                                 I/O Timings: read=1.514 write=0.000
         ->  Index Scan using ml_model_version_metadata_pkey on public.ml_model_version_metadata  (cost=0.29..3.31 rows=1 width=14) (never executed)
               Index Cond: (ml_model_version_metadata.id = "ANY_subquery".id)
               I/O Timings: read=0.000 write=0.000
ml_model_versions (1,000 rows, project 72033045, 50 rows, candidates and version metadata already deleted)
DELETE FROM "ml_model_versions" WHERE "ml_model_versions"."id" IN (SELECT "ml_model_versions"."id" FROM "ml_model_versions" WHERE "ml_model_versions"."project_id" = 72033045 LIMIT 1000)

cold:

 Delete on public.ml_model_versions  (cost=48.56..191.53 rows=0 width=0) (actual time=10.310..10.312 rows=0 loops=1)
   Buffers: shared hit=262 read=16 dirtied=9
   WAL: records=50 fpi=8 bytes=67776
   I/O Timings: read=9.802 write=0.000
   ->  Nested Loop  (cost=48.56..191.53 rows=50 width=38) (actual time=6.451..8.599 rows=50 loops=1)
         Buffers: shared hit=149 read=14
         I/O Timings: read=8.329 write=0.000
         ->  HashAggregate  (cost=48.27..48.77 rows=50 width=40) (actual time=5.846..5.857 rows=50 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 24kB
               Buffers: shared hit=3 read=10
               I/O Timings: read=5.701 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.29..48.15 rows=50 width=40) (actual time=2.238..5.826 rows=50 loops=1)
                     Buffers: shared hit=3 read=10
                     I/O Timings: read=5.701 write=0.000
                     ->  Limit  (cost=0.29..47.65 rows=50 width=8) (actual time=2.226..5.800 rows=50 loops=1)
                           Buffers: shared hit=3 read=10
                           I/O Timings: read=5.701 write=0.000
                           ->  Index Scan using index_ml_model_versions_on_project_id on public.ml_model_versions ml_model_versions_1  (cost=0.29..47.65 rows=50 width=8) (actual time=2.225..5.793 rows=50 loops=1)
                                 Index Cond: (ml_model_versions_1.project_id = 72033045)
                                 Buffers: shared hit=3 read=10
                                 I/O Timings: read=5.701 write=0.000
         ->  Index Scan using ml_model_versions_pkey on public.ml_model_versions  (cost=0.29..2.85 rows=1 width=14) (actual time=0.054..0.054 rows=1 loops=50)
               Index Cond: (ml_model_versions.id = "ANY_subquery".id)
               Buffers: shared hit=146 read=4
               I/O Timings: read=2.628 write=0.000
Trigger RI_ConstraintTrigger_a_40027 for constraint fk_e86e0bfa5a: time=1.474 calls=50
Trigger RI_ConstraintTrigger_a_43681 for constraint fk_rails_5b67cc9107: time=4.235 calls=50
ml_model_metadata (1,000 rows, project 72033045)
DELETE FROM "ml_model_metadata" WHERE "ml_model_metadata"."id" IN (SELECT "ml_model_metadata"."id" FROM "ml_model_metadata" WHERE "ml_model_metadata"."project_id" = 72033045 LIMIT 1000)

cold:

 Delete on public.ml_model_metadata  (cost=3.31..6.40 rows=0 width=0) (actual time=0.668..0.671 rows=0 loops=1)
   Buffers: shared hit=3 read=1
   I/O Timings: read=0.628 write=0.000
   ->  Nested Loop  (cost=3.31..6.40 rows=1 width=38) (actual time=0.667..0.669 rows=0 loops=1)
         Buffers: shared hit=3 read=1
         I/O Timings: read=0.628 write=0.000
         ->  HashAggregate  (cost=3.17..3.18 rows=1 width=40) (actual time=0.666..0.668 rows=0 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 24kB
               Buffers: shared hit=3 read=1
               I/O Timings: read=0.628 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.14..3.17 rows=1 width=40) (actual time=0.664..0.665 rows=0 loops=1)
                     Buffers: shared hit=3 read=1
                     I/O Timings: read=0.628 write=0.000
                     ->  Limit  (cost=0.14..3.16 rows=1 width=8) (actual time=0.663..0.664 rows=0 loops=1)
                           Buffers: shared hit=3 read=1
                           I/O Timings: read=0.628 write=0.000
                           ->  Index Scan using index_ml_model_metadata_on_project_id on public.ml_model_metadata ml_model_metadata_1  (cost=0.14..3.16 rows=1 width=8) (actual time=0.662..0.663 rows=0 loops=1)
                                 Index Cond: (ml_model_metadata_1.project_id = 72033045)
                                 Buffers: shared hit=3 read=1
                                 I/O Timings: read=0.628 write=0.000
         ->  Index Scan using ml_model_metadata_pkey on public.ml_model_metadata  (cost=0.14..3.16 rows=1 width=14) (never executed)
               Index Cond: (ml_model_metadata.id = "ANY_subquery".id)
               I/O Timings: read=0.000 write=0.000
ml_models (1,000 rows, project 74016851, 1,165 models; versions and model metadata already deleted, 1,178 experiments and 5,548 candidates still present, so the trigger time is an upper bound)
DELETE FROM "ml_models" WHERE "ml_models"."id" IN (SELECT "ml_models"."id" FROM "ml_models" WHERE "ml_models"."project_id" = 74016851 LIMIT 1000)

cold:

 Delete on public.ml_models  (cost=480.47..1601.30 rows=0 width=0) (actual time=188.697..188.700 rows=0 loops=1)
   Buffers: shared hit=5304 read=305 dirtied=294
   WAL: records=1000 fpi=293 bytes=2036913
   I/O Timings: read=182.874 write=0.000
   ->  Nested Loop  (cost=480.47..1601.30 rows=1000 width=38) (actual time=174.836..186.535 rows=1000 loops=1)
         Buffers: shared hit=3001 read=305
         I/O Timings: read=182.874 write=0.000
         ->  HashAggregate  (cost=480.18..490.18 rows=1000 width=40) (actual time=174.067..174.252 rows=1000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 193kB
               Buffers: shared hit=13 read=293
               I/O Timings: read=171.980 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.29..477.68 rows=1000 width=40) (actual time=0.774..173.629 rows=1000 loops=1)
                     Buffers: shared hit=13 read=293
                     I/O Timings: read=171.980 write=0.000
                     ->  Limit  (cost=0.29..467.68 rows=1000 width=8) (actual time=0.758..173.272 rows=1000 loops=1)
                           Buffers: shared hit=13 read=293
                           I/O Timings: read=171.980 write=0.000
                           ->  Index Scan using index_ml_models_on_project_id on public.ml_models ml_models_1  (cost=0.29..893.95 rows=1912 width=8) (actual time=0.757..173.163 rows=1000 loops=1)
                                 Index Cond: (ml_models_1.project_id = 74016851)
                                 Buffers: shared hit=13 read=293
                                 I/O Timings: read=171.980 write=0.000
         ->  Index Scan using ml_models_pkey on public.ml_models  (cost=0.29..1.19 rows=1 width=14) (actual time=0.012..0.012 rows=1 loops=1000)
               Index Cond: (ml_models.id = "ANY_subquery".id)
               Buffers: shared hit=2988 read=12
               I/O Timings: read=10.894 write=0.000
Trigger RI_ConstraintTrigger_a_37290 for constraint fk_4e8b59e7a8: time=4.889 calls=1000
Trigger RI_ConstraintTrigger_a_45473 for constraint fk_rails_97194a054e: time=17.614 calls=1000
Trigger RI_ConstraintTrigger_a_46935 for constraint fk_rails_d907835e01: time=5.685 calls=1000
Trigger RI_ConstraintTrigger_a_37415 for constraint fk_56d6ed4d3d: time=36.209 calls=1000
Trigger RI_ConstraintTrigger_a_44359 for constraint fk_rails_6b39844d44: time=4.604 calls=1000
Late batches on ml_candidate_metrics (project 72033045): 1,000-row statement after 500,000 rows of the project were deleted
DELETE FROM "ml_candidate_metrics" WHERE "ml_candidate_metrics"."id" IN (SELECT "ml_candidate_metrics"."id" FROM "ml_candidate_metrics" WHERE "ml_candidate_metrics"."project_id" = 72033045 LIMIT 1000)

first statement after the 500,000 deletions (walks the dead index entries):

 Delete on public.ml_candidate_metrics  (cost=674.58..4262.02 rows=0 width=0) (actual time=248.722..248.724 rows=0 loops=1)
   Buffers: shared hit=30843 read=74 dirtied=642
   WAL: records=25190 fpi=642 bytes=6591358
   I/O Timings: read=9.921 write=0.000
   ->  Nested Loop  (cost=674.58..4262.02 rows=1000 width=38) (actual time=240.503..247.827 rows=1000 loops=1)
         Buffers: shared hit=29777 read=74 dirtied=576
         WAL: records=24190 fpi=576 bytes=5998724
         I/O Timings: read=9.921 write=0.000
         ->  HashAggregate  (cost=674.02..684.02 rows=1000 width=40) (actual time=240.464..240.681 rows=1000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 193kB
               Buffers: shared hit=24784 read=67 dirtied=576
               WAL: records=24190 fpi=576 bytes=5998724
               I/O Timings: read=5.004 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.56..671.52 rows=1000 width=40) (actual time=234.356..240.031 rows=1000 loops=1)
                     Buffers: shared hit=24784 read=67 dirtied=576
                     WAL: records=24190 fpi=576 bytes=5998724
                     I/O Timings: read=5.004 write=0.000
                     ->  Limit  (cost=0.56..661.52 rows=1000 width=8) (actual time=234.336..239.828 rows=1000 loops=1)
                           Buffers: shared hit=24784 read=67 dirtied=576
                           WAL: records=24190 fpi=576 bytes=5998724
                           I/O Timings: read=5.004 write=0.000
                           ->  Index Scan using index_ml_candidate_metrics_on_project_id on public.ml_candidate_metrics ml_candidate_metrics_1  (cost=0.56..602287.47 rows=911244 width=8) (actual time=234.333..239.740 rows=1000 loops=1)
                                 Index Cond: (ml_candidate_metrics_1.project_id = 72033045)
                                 Buffers: shared hit=24784 read=67 dirtied=576
                                 WAL: records=24190 fpi=576 bytes=5998724
                                 I/O Timings: read=5.004 write=0.000
         ->  Index Scan using ml_candidate_metrics_pkey on public.ml_candidate_metrics  (cost=0.56..3.58 rows=1 width=14) (actual time=0.007..0.007 rows=1 loops=1000)
               Index Cond: (ml_candidate_metrics.id = "ANY_subquery".id)
               Buffers: shared hit=4993 read=7
               I/O Timings: read=4.917 write=0.000

next statement (the previous scan marked the dead entries):

 Delete on public.ml_candidate_metrics  (cost=674.58..4262.02 rows=0 width=0) (actual time=11.203..11.206 rows=0 loops=1)
   Buffers: shared hit=6717 read=67 dirtied=65
   WAL: records=1001 fpi=65 bytes=583197
   I/O Timings: read=3.570 write=0.000
   ->  Nested Loop  (cost=674.58..4262.02 rows=1000 width=38) (actual time=6.999..10.436 rows=1000 loops=1)
         Buffers: shared hit=5653 read=67 dirtied=1
         WAL: records=1 fpi=1 bytes=7533
         I/O Timings: read=3.570 write=0.000
         ->  HashAggregate  (cost=674.02..684.02 rows=1000 width=40) (actual time=6.960..7.122 rows=1000 loops=1)
               Group Key: "ANY_subquery".id
               Batches: 1  Memory Usage: 193kB
               Buffers: shared hit=655 read=65 dirtied=1
               WAL: records=1 fpi=1 bytes=7533
               I/O Timings: read=2.131 write=0.000
               ->  Subquery Scan on "ANY_subquery"  (cost=0.56..671.52 rows=1000 width=40) (actual time=3.837..6.701 rows=1000 loops=1)
                     Buffers: shared hit=655 read=65 dirtied=1
                     WAL: records=1 fpi=1 bytes=7533
                     I/O Timings: read=2.131 write=0.000
                     ->  Limit  (cost=0.56..661.52 rows=1000 width=8) (actual time=3.824..6.528 rows=1000 loops=1)
                           Buffers: shared hit=655 read=65 dirtied=1
                           WAL: records=1 fpi=1 bytes=7533
                           I/O Timings: read=2.131 write=0.000
                           ->  Index Scan using index_ml_candidate_metrics_on_project_id on public.ml_candidate_metrics ml_candidate_metrics_1  (cost=0.56..602287.47 rows=911244 width=8) (actual time=3.822..6.447 rows=1000 loops=1)
                                 Index Cond: (ml_candidate_metrics_1.project_id = 72033045)
                                 Buffers: shared hit=655 read=65 dirtied=1
                                 WAL: records=1 fpi=1 bytes=7533
                                 I/O Timings: read=2.131 write=0.000
         ->  Index Scan using ml_candidate_metrics_pkey on public.ml_candidate_metrics  (cost=0.56..3.58 rows=1 width=14) (actual time=0.003..0.003 rows=1 loops=1000)
               Index Cond: (ml_candidate_metrics.id = "ANY_subquery".id)
               Buffers: shared hit=4998 read=2
               I/O Timings: read=1.439 write=0.000

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Eduardo Bonet

Merge request reports

Loading
Loading