Add metrics and event tracking for Protected Container Repositories

Tim Rizzirequested to merge
add-container-registry-protection-repository-metrics into master

Overview

This MR adds usage metrics and internal event tracking for Protected Container Repositories feature. This is the first of three MRs to split container registry protection instrumentation into smaller, focused changes.

What's Changed

Metrics Added (CE - tracks usage on Free, Premium, and Ultimate tiers)

  1. counts.projects_with_container_registry_protection_rules
    • Count of distinct projects that have at least one container repository protection rule
    • Instrumentation: CountProjectsWithContainerRegistryProtectionRulesMetric
  2. counts.container_registry_protection_rules
    • Total count of all container repository protection rules across all projects
    • Instrumentation: CountContainerRegistryProtectionRulesMetric
  3. counts.distinct_top_level_groups_with_container_registry_protection_rules
    • Count of distinct top-level groups (customers) that have projects with container repository protection rules
    • Instrumentation: CountDistinctTopLevelGroupsWithContainerRegistryProtectionRulesMetric

Event Tracking

  1. create_container_repository_protection_rule
    • Tracked in ContainerRegistry::Protection::CreateRuleService
    • Includes context: project, namespace, user
  2. delete_container_repository_protection_rule
    • Tracked in ContainerRegistry::Protection::DeleteRuleService
    • Includes context: project, namespace, user

Files Changed

  • Metrics: 3 new metric instrumentation classes (CE - in lib/)
  • Metric Definitions: 3 new metric YAML files (EE - metadata only)
  • Events: 2 new event YAML definitions
  • Service Classes: Added event tracking to CreateRuleService and DeleteRuleService
  • Specs: Added tests for metrics and event tracking (CE - in spec/)

Testing

  • ✅ Metric instrumentation specs added
  • ✅ Event tracking specs added in service specs
  • ✅ All existing tests pass

Database Review

This MR adds three new database metrics for container repository protection rules. Below are the SQL queries and query plans for database review.

Query 1: counts.container_registry_protection_rules

Description: Total count of container repository protection rules

SQL Query:

SELECT COUNT("container_registry_protection_rules"."id")
FROM "container_registry_protection_rules";

Query Plan:

Summary:

Query 2: counts.projects_with_container_registry_protection_rules

Description: Count of distinct projects with container repository protection rules

SQL Query:

SELECT COUNT(DISTINCT "container_registry_protection_rules"."project_id")
FROM "container_registry_protection_rules";

Query Plan:

Summary:

Query 3: counts.distinct_top_level_groups_with_container_registry_protection_rules

Description: Count of distinct top-level groups with container repository protection rules

SQL Query:

SELECT
    COUNT(DISTINCT "namespaces"."id")
FROM
    "namespaces"
    INNER JOIN "projects" ON "projects"."namespace_id" = "namespaces"."id"
    INNER JOIN "container_registry_protection_rules" ON "container_registry_protection_rules"."project_id" = "projects"."id"
WHERE
    "namespaces"."type" = 'Group'
    AND "namespaces"."parent_id" IS NULL

Query Plan:

Full Execution Plan 
 Aggregate  (cost=2520.80..2520.81 rows=1 width=8) (actual time=1477.986..1477.989 rows=1 loops=1)
   Buffers: shared hit=3466 read=1887 dirtied=21
   WAL: records=24 fpi=21 bytes=164142
   I/O Timings: read=1448.784 write=0.000
   ->  Sort  (cost=2520.66..2520.73 rows=29 width=4) (actual time=1477.943..1477.958 rows=247 loops=1)
         Sort Key: namespaces.id
         Sort Method: quicksort  Memory: 25kB
         Buffers: shared hit=3466 read=1887 dirtied=21
         WAL: records=24 fpi=21 bytes=164142
         I/O Timings: read=1448.784 write=0.000
         ->  Nested Loop  (cost=1.28..2519.95 rows=29 width=4) (actual time=6.546..1477.751 rows=247 loops=1)
               Buffers: shared hit=3463 read=1887 dirtied=21
               WAL: records=24 fpi=21 bytes=164142
               I/O Timings: read=1448.784 write=0.000
               ->  Nested Loop  (cost=0.85..2194.05 rows=712 width=4) (actual time=3.875..1156.932 rows=712 loops=1)
                     Buffers: shared hit=1663 read=1378 dirtied=16
                     WAL: records=19 fpi=16 bytes=124405
                     I/O Timings: read=1137.844 write=0.000
                     ->  Index Only Scan using i_container_protection_unique_project_repository_path_pattern on public.container_registry_protection_rules  (cost=0.28..27.45 rows=712 width=8) (actual time=0.026..4.525 rows=712 loops=1)
                           Heap Fetches: 0
                           Buffers: shared hit=4 read=7
                           I/O Timings: read=3.802 write=0.000
                     ->  Memoize  (cost=0.57..3.59 rows=1 width=8) (actual time=1.617..1.617 rows=1 loops=712)
                           Buffers: shared hit=1659 read=1371 dirtied=16
                           WAL: records=19 fpi=16 bytes=124405
                           I/O Timings: read=1134.042 write=0.000
                           ->  Index Scan using projects_pkey on public.projects  (cost=0.56..3.58 rows=1 width=8) (actual time=1.899..1.899 rows=1 loops=605)
                                 Index Cond: (projects.id = container_registry_protection_rules.project_id)
                                 Buffers: shared hit=1657 read=1371 dirtied=14
                                 WAL: records=17 fpi=14 bytes=109447
                                 I/O Timings: read=1134.042 write=0.000
               ->  Index Only Scan using index_groups_on_parent_id_id on public.namespaces  (cost=0.43..0.46 rows=1 width=4) (actual time=0.449..0.449 rows=0 loops=712)
                     Index Cond: ((namespaces.parent_id IS NULL) AND (namespaces.id = projects.namespace_id))
                     Heap Fetches: 28
                     Buffers: shared hit=1800 read=509 dirtied=5
                     WAL: records=5 fpi=5 bytes=39737
                     I/O Timings: read=310.940 write=0.000
Settings: work_mem = '100MB', random_page_cost = '1.5', seq_page_cost = '4', effective_cache_size = '472585MB', jit = 'off'
Summary 
Time: 1.486 s  
  - planning: 7.516 ms  
  - execution: 1.478 s  
    - I/O read: 1.449 s  
    - I/O write: 0.000 ms  
  
Shared buffers:  
  - hits: 3466 (~27.10 MiB) from the buffer pool  
  - reads: 1887 (~14.70 MiB) from the OS file cache, including disk I/O  
  - dirtied: 21 (~168.00 KiB)  
  - writes: 0

Notes

  • All three metrics use time_frame: all, so they count all records without time constraints
  • The queries are executed as part of Service Ping collection
  • The container_registry_protection_rules table is a relatively new table (introduced in 16.5) and is expected to be small initially
  • Query 3 uses array indexing on traversal_ids[1] to get the root namespace, which is a common pattern in GitLab for hierarchical namespace structures
  • All queries should be tested with actual data from GitLab.com production (using appropriate project/namespace IDs as mentioned in the database review guidelines)

Related MRs

This MR is part of a series:

  • MR 1 (this MR): Protected Container Repositories
  • MR 2: Protected Container Tags (mutable) - !213723 (merged)
  • MR 3: Immutable Container Tags - !213725

Feature Information

  • Feature: Container Repository Protection Rules
  • Feature Availability: Free, Premium, Ultimate (CE feature)
  • Metrics Tracking: CE (tracks usage on all tiers)
  • Product Group: Package Registry
  • Product Category: Container Registry
Edited by Adie (she/her)

Merge request reports