Implement deleted objects for expired artifacts
What does this MR do?
Related issues: https://gitlab.com/gitlab-org/gitlab/-/issues/233939, https://gitlab.com/gitlab-org/gitlab/-/issues/223034, #220422 (closed)
This merge request builds on top of !39464 (merged) to efficiently destroy expired job artifacts by:
- copying object storage attributes from
ci_job_artifacts
intoci_deleted_objects
- aggregating
project_statistics
updates for each project. If we have more than one artifact from a project, it will do a single update for that project - deleting job_artifacts with a single SQL query.
It also fixes #267529 (closed) because it touches the same code.
SQL queries:
- Mass insert: https://explain.depesz.com/s/RmWn - I don't think this is a relevant plan because it uses
a
asstore_dir
andb
asfile
to get around the Slack limitation for message size:Your message is 2234 characters too long
- Mass delete from
ci_job_artifacts
: https://explain.depesz.com/s/xmmR - Mass delete from
security_findings
: https://explain.depesz.com/s/Tl77 - the selected ids don't have any security findings
Screenshots
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team