Skip to content

Add worker to cleanup orphaned packages_nuget_symbols records

Context

In Add Packages::Nuget::Symbol model and correspon... (!129916 - merged), we introduced a new database-backed model Packages::Nuget::Symbol that's used to store the indexed NuGet symbol files. Every record in the packages_nuget_symbols table is linked to a parent package using the package_id foreign key.

If the parent package got deleted from the database, we nullify this package_id foreign key in the packages_nuget_symbols table, and thus the records with package_id as null are considered stale entries, and should be deleted.

In this MR, we clean up those stale packages_nuget_symbols entries using a limited capacity worker.

What does this MR do and why?

This MR introduces a new limited capacity background worker Packages::Nuget::CleanupStaleSymbolsWorker to delete stale Nuget symbols entries.

The existing Packages::CleanupPackageRegistryWorker will enqueue Packages::Nuget::CleanupStaleSymbolsWorker if there are any stale Nuget symbols entries.

Screenshots or screen recordings

N/A

How to set up and validate locally

  1. In the rails console Create two Nuget symbol records and mark one of them as stale; i.e its package_id = nil

    def fixture_file_upload(*args, **kwargs)
      Rack::Test::UploadedFile.new(*args, **kwargs)
    end
    
    package = FactoryBot.create(:nuget_package, name: 'test', project: Project.first)
    
    symbol1 = FactoryBot.create(:nuget_symbol, package_id: package.id)
    symbol2 = FactoryBot.create(:nuget_symbol, :stale, package_id: package.id)
  2. Make sure sidekiq is up and running otherwise gdk restart rails-background-jobs

    Packages::CleanupPackageRegistryWorker.new.perform
  3. After the worker has been executed, try the following queries:

     # symbol not deleted because it's not stale
     Packages::Nuget::Symbol.find(symbol1.id)
     => #<Packages::Nuget::Symbol ...  
    
     # deleted as it's stale
     Packages::Nuget::Symbol.find(symbol2.id)
     => ActiveRecord::RecordNotFound: Couldn't find Packages::Nuget::Symbol 

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #424336 (closed)

Edited by Moaz Khalifa

Merge request reports