Follow-up from "Add cron worker for cleaning up unit test tables"
The following discussion from !61463 (merged) should be addressed:
This query has a nested loop inside (
NOT EXISTS
) which might time out if we have a lot's ofci_unit_test_failures
records.The linked plan shows that we scanned 5,238 rows to get 100 deletable UnitTest records back: https://explain-depesz.postgres.ai/s/bv
The query execution time will grow as we scan more records in the DB. We might need to alter the strategy later. Options:
- Add
has_test_failure
row tounit_tests
(de-normalization), in order to make the scan work on a single table.- Iterating over the table (EachBatch) and use
NOT EXISTS
within the block. We call this the slow iteration approach: https://docs.gitlab.com/ee/development/iterating_tables_in_batches.html#slow-iteration
Look into adding the has_test_failure
column.