Skip to content

Add concern for calculating namespace statistics

Vijay Hawoldar requested to merge vij-dp-statistics into master

What does this MR do and why?

We have recently added dependency_proxy_size to NamespaceStatistics and Namespaces::RootStorageStatistics.

We need to trigger an update of those columns anytime the relevant Dependency Proxy assets change (DependenyProxy::Manifests and DependencyProxy::Blobs), i.e. when they're added, updated or deleted.

To achieve that in this MR, I have added a new concern that can be added to different models and allow customisable updates to the NamespaceStatistics. I opted for a concern to reduce duplication for DependencyProxy assets, but also because we're likely to want to do the same thing for other storage related assets in the near future (e.g. Container Registry)

How to set up and validate locally

  1. Check the namespace statistics values for an existing group:

      pp my_group.namespace_statistics
  2. Add some DependencyProxy records with a size value assigned

      dpm =  dpm = DependencyProxy::Manifest.new(group: my_group, size: 100, file_name: 'abc.txt', file: 'file', digest: 'abc123')
      dpm.save(validate: false) # bypasses file validation issues which we don't need for this verification
    
      dpb = DependencyProxy::Blob.new(group: my_group, size: 100, file: 'file', file_name: 'file_name.txt')
      dpb.save(validate: false) # bypasses file validation issues which we don't need for this verification
  3. Check the namespace statistics values for an existing group:

      pp my_group.namespace_statistics.reload 
      # dependency_proxy_size and storage_size should have increased by `200`
      # this may take some time because it's processed via a sidekiq job
  4. Check that the Namespaces::RootStatisticsWorker was scheduled in http://gdk.host:3000/admin/sidekiq/scheduled

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Vijay Hawoldar

Merge request reports