Skip to content

Extract batch destroy Service from Job Artifact Expiration [RUN ALL RSPEC]

Allison Browne requested to merge refactor-parallel-destroy-service into master

What does this MR do?

This MR is a pure refactor which extracts a parallel batch destroy service out of the destroy expired job artifacts service.

This is done in order to create a service that batch destroys artifacts when a project is deleted, in order to speed up project deletion. As part of the re-architect artifacts work there is already a service that deletes the files asynchronously in object storage once they are placed in the ci_deleted_objects table.

For Job Artifacts, if we add project related artifacts to ci_deleted_objects and delete the files async then we can remove dependent: destroy from builds (which causes an N+1 issue). Currently, we need dependent: destroy so that callbacks that delete the artifact files are executed. Then we can use database level cascades for all build-related relations without callbacks and fast_destroy_all for all build-related relations with callbacks.

build dependant relationships:

relationship has_callbacks?
deployment Yes - already FastDestroyAll
pending_state no
trace_sections no
trace_chunks Yes - already FastDestroyAll
report_results no
job_artifacts Yes
job_variables no
sourced_pipelines no
pages_deployments no
runner_session no

Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/24644

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team
Edited by Allison Browne

Merge request reports