Clean up Conan objects when deleting package file

• Please check this box if this contribution uses AI-generated content (including content generated by GitLab Duo features) as outlined in the GitLab DCO & CLA. As a benefit of being a GitLab Community Contributor, you receive complimentary access to GitLab Duo.

What does this MR do and why?

Clean up Conan objects when deleting Conan package file

• Introduced a before_destroy method in both CleanupArtifactWorker and CleanupPackageFileWorker to handle pre-destruction logic.
• Implemented cleanup logic for Conan package files, ensuring that associated recipe revisions, package references, and package revisions are deleted when no longer needed.

Context

While working on improving the UI for Conan packages, I noticed it is possible to delete the files in the UI. By doing so, we can now delete all files in a recipe revision, package reference or package revision without deleting the linked Conan objects. The idea of this MR is to fix that, so that we clean up the linked recipe revision, package reference and package revision if it is the last file like we do for the package.

Note: this can only happen when deleting via the UI or the graphql api as when deleting via the Conan api, we are deleting the whole package, or revision.

References

Screenshots or screen recordings

Before	After

How to set up and validate locally

Test setup

To test the worker, we to create a test setup:

1. Set up the feature flag
2. Get a Conan authentication token
3. Upload 4 files with different revisions

1. Feature Flag Setup

For GDK, open rails console (docs):

gdk rails console

Enable/disable the feature flag:

# Enable
Feature.enable(:conan_package_revisions_support)

2. Get Conan Authentication Token

First, generate a base64-encoded Basic Auth token:

# Replace with your username and PAT
echo -n "USERNAME:glpat-YOUR-TOKEN" | base64

Then use it to get a Conan JWT token:

curl --request GET \
  --url 'http://localhost:3000/api/v4/projects/1/packages/conan/v1/users/authenticate' \
  --header 'Authorization: Basic <YOUR-BASE64-TOKEN>' \
  --header 'Accept: text/plain'

Upload files

Upload 2 recipe files with different recipe revisions

echo "class UploadV2Package:" > conanfile.py

curl --request PUT \
  --form 'file=@conanfile.py' \
'http://localhost:3000/api/v4/projects/<project_id>/packages/conan/v2/conans/worker-test/1.0.0/user/stable/revisions/00000000000000000000000000000001/files/conanfile.py' \
  --header 'Authorization: Bearer <YOUR-CONAN-JWT-TOKEN>'

curl --request PUT \
  --form 'file=@conanfile.py' \
'http://localhost:3000/api/v4/projects/<project_id>/packages/conan/v2/conans/worker-test/1.0.0/user/stable/revisions/00000000000000000000000000000002/files/conanfile.py' \
  --header 'Authorization: Bearer <YOUR-CONAN-JWT-TOKEN>'

Upload 2 package files with different package revisions

echo "info" > conaninfo.txt

curl --request PUT \
  --form 'file=@conaninfo.txt' \
'http://localhost:3000/api/v4/projects/<project_id>/packages/conan/v2/conans/worker-test/1.0.0/user/stable/revisions/00000000000000000000000000000001/packages/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/revisions/00000000000000000000000000000001/files/conaninfo.txt' \
  --header 'Authorization: Bearer <YOUR-CONAN-JWT-TOKEN>'

curl --request PUT \
  --form 'file=@conaninfo.txt' \
'http://localhost:3000/api/v4/projects/<project_id>/packages/conan/v2/conans/worker-test/1.0.0/user/stable/revisions/00000000000000000000000000000001/packages/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/revisions/00000000000000000000000000000002/files/conaninfo.txt' \
  --header 'Authorization: Bearer <YOUR-CONAN-JWT-TOKEN>'

Test Worker

Go to rails console:

gdk rails console

Check up the data, you should have 4 files, 2 recipe revisions, 1 package reference and 2 package revisions

package = Packages::Package.find_by(name: 'worker-test', version: '1.0.0')
package.installable_package_files
package.conan_recipe_revisions
package.conan_package_revisions
package.conan_package_references

Step 1: Test package revision deletion

Get the second package file, linked to package_revision 00000000000000000000000000000002 and mark it as pending destruction

finder = Packages::Conan::PackageFileFinder.new(
  package,
  file_name: 'conaninfo.txt',
  conan_file_type: 'package_file',
  recipe_revision: '00000000000000000000000000000001',
  conan_package_reference: '5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9',
  package_revision: '00000000000000000000000000000002'
)
package_file = finder.execute
package_file.update!(status: :pending_destruction)

Run the worker:

Packages::CleanupPackageFileWorker.perform_sync

The package should now only have one package revision

package.reload.conan_package_revisions

Step 2: Test package reference deletion

Get the last package file, linked to package_revision 00000000000000000000000000000001 and mark it as pending destruction

finder = Packages::Conan::PackageFileFinder.new(
  package,
  file_name: 'conaninfo.txt',
  conan_file_type: 'package_file',
  recipe_revision: '00000000000000000000000000000001',
  conan_package_reference: '5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9',
  package_revision: '00000000000000000000000000000001'
)
package_file = finder.execute
package_file.update!(status: :pending_destruction)

Run the worker:

Packages::CleanupPackageFileWorker.perform_sync

The package should now have no package revision and no package reference.

package.reload.conan_package_revisions
package.conan_package_references

Step 3: Test recipe revision deletion

Get the first recipe file, linked to recipe_revision 00000000000000000000000000000002 and mark it as pending destruction

finder = Packages::Conan::PackageFileFinder.new(
  package,
  file_name: 'conanfile.py',
  conan_file_type: 'recipe_file',
  recipe_revision: '00000000000000000000000000000001'
)
package_file = finder.execute
package_file.update!(status: :pending_destruction)

Run the worker:

Packages::CleanupPackageFileWorker.perform_sync

The package should now only have one recipe revision

package.reload.conan_recipe_revisions

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #519741 (closed)

/cc @mbo5be