Improve performance of expire build artifacts worker
requested to merge improve-performance-of-expire-build-artifacts-worker into cleanup-leagcy-artifact-migration
What does this MR do?
This MR improves the performance of ExpireBuildArtifactsWorker
, which is currently unable to run because of statement time on gitlab.com scale.
Mainly, there are two improvements.
- Scope out legacy artifacts from the query as it's already been migrated to new artifacts
- Use
EachBatch
to efficiently walkthrough giganticci_builds
table (over 100M rows)
What are the relevant issue numbers?
Close https://gitlab.com/gitlab-org/gitlab-ce/issues/41057
Does this MR meet the acceptance criteria?
-
Changelog entry added, if necessary -
Documentation created/updated -
Tests added for this feature/bug -
Tested in all supported browsers -
Conforms to the code review guidelines -
Conforms to the merge request performance guidelines -
Conforms to the style guides -
Conforms to the database guides -
Link to e2e tests MR added if this MR has Requires e2e tests label. See the Test Planning Process. -
Security reports checked/validated by reviewer
Edited by Shinya Maeda