Refactor NamespaceStatistics into CE
What does this MR do and why?
Context
As part of https://gitlab.com/groups/gitlab-org/-/epics/7011 we are going to begin displaying storage usage for certain containers that were previously unexposed, starting with Dependency Proxy in https://gitlab.com/gitlab-org/gitlab/-/issues/348172
To achieve this, we're re-using the existing storage calculations at the Namespace level via NamespaceStatistics
and calculating the total across all namespaces in a hierarchy via Namespace::RootStorageStatistics
The existing NamespaceStatistics
implementation is within the EE namespace because at the time of creation, the only namespace (group) level statistic being tracked was group wiki size, which is a Premium+ feature.
Dependency Proxy is available for groups on all tiers.
This MR
So that we can calculate and store dependency proxy totals at the namespace and root namespace statistic levels for CE, this MR makes the following changes:
- move most of the
NamespaceStatistic
implementation into a new CE class of the same name, - convert the old
NamespaceStatistic
class into an EE module to cater for the group wiki aspect - add
dependency_proxy_size
calculations to theNamespaceStatistic
refresh - move the
NamespaceStatistic
calculations from the EERootStorageStatistic
to the CE version - add
NamespaceStatistic#dependency_proxy_size
to theRootStorageStatistic
calculations
I intend to handle the update of NamespaceStatistics
when dependency proxy assets are added/removed in a separate MR.
How to set up and validate locally
- Create some dependency proxy records with a
size
in the DB:dpm = DependencyProxy::Manifest.new(size: 100, group: my_group) dpb = DependencyProxy::Blob.new(size: 200, group: my_group) dpm.save(validate: false) # bypasses the need to satisfy DP:M validations dpb.save(validate: false) # bypasses the need to satisfy DP:B validations
- Check storage sizes and trigger a refresh
namespace_statistics = NamespaceStatistics.find_or_create_by!(namespace_id: group.id) namespace_statistics.storage_size namespace_statistics.dependency_proxy_size namespace_statistics.refresh! namespace_statistics.storage_size # expect 300 namespace_statistics.dependency_proxy_size # expect 300
- Check root storage statistics and trigger a refresh
root_statistics = Namespace::RootStorageStatistics.find_or_create_by!(namespace_id: group.id) root_statistics.dependency_proxy_size root_statistics.recalculate! root_statistics.dependency_proxy_size # expect 300
If you have a group on a premium or ultimate plan, you could optionally create a group wiki and re-run the checks above but also check wiki_size
. It's worth noting that at the RootStorageStatistics#wiki_size
will also include project level wikis.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.