Delete part uploads before destroying dependency list exports

Ryan Wellsrequested to merge
fix/dependency-list-export-part-orphaned-uploads into master

What does this MR do and why?

Ensures that all Upload records associated with Dependency Exports are correctly, and fully, deleted when the overall export is expired.

This MR makes Sbom::DeleteExpiredExportsWorker delete each batch's export parts and their uploads before deleting the exports. Upload.destroy_for_associations! also removes the underlying files from object storage via begin_fast_destroy / finalize_fast_destroy.

This is the same defect reported for Vulnerabilities::Export::Part in #602878 (closed), fixed in !254286 (merged).

Dependencies::DestroyExportWorker is not affected by this bug. It calls destroy!, which runs the dependent: :destroy association on export_parts and CarrierWave's normal upload cleanup, so that path was already correct.

Not covered by this MR

  • dependency_list_exports also has loose foreign keys with on_delete: async_delete for pipelines, users, projects, and namespaces. Loose foreign key cleanup issues a raw DELETE, so deleting a project still cascades the parts away and orphans both the export and the part uploads.
  • Unlike vulnerability_exports, dependency_list_exports is not registered in Gitlab::Database::TablesWithDestroyServices, so there is no foreign_keys_to_destroy_service_tables_spec guard to keep this from regressing. Adding one would mean extracting a destroy service, which is out of scope here.
  • This MR does not clean up orphans that already exist. Those need the documented delete_orphaned_uploads console step: https://docs.gitlab.com/administration/geo/replication/troubleshooting/synchronization_verification/#failed-verification-of-uploads-on-the-primary-geo-site

References

How to set up and validate locally

  1. Open a rails console: bundle exec rails console.
  2. Run the following. On master, the last line returns 1; with this MR it returns 0.
project = Project.last
author = User.last
export = FactoryBot.create(:dependency_list_export, :with_file, expires_at: 1.hour.ago, project: project, author: author)
part = FactoryBot.create(:dependency_list_export_part, :exported, dependency_list_export: export)
Upload.for_model_type_and_id(Dependencies::DependencyListExport::Part, part.id).count # => 1
Sbom::DeleteExpiredExportsWorker.new.perform
Dependencies::DependencyListExport::Part.exists?(part.id) # => false
Upload.for_model_type_and_id(Dependencies::DependencyListExport::Part, part.id).count # => 0

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Ryan Wells

Merge request reports