Add TrackedContextQuotaService for org-level quota enforcement

What does this MR do and why?

Implements Security::TrackedContextQuotaService to enforce organization-level quotas on tracked security contexts during the Vulnerabilities Across Contexts (VAC) closed beta rollout.

The quota limit is read from Organizations::OrganizationSetting#security_tracked_context_quota_with_default (added in !233389 (merged)). Enforcement is gated by its own :security_tracked_context_quota_enforcement feature flag, independently of VAC for now. This lets us test and roll out quota enforcement on its own, verify it behaves well, and only then bind it into the VAC rollout as a follow-up (so VAC-enabled organizations get enforcement together once it is proven).

Changes

  1. New service ee/app/services/security/tracked_context_quota_service.rb:
    • quota_available?(actor) - true unless enforcement is enabled (per the :security_tracked_context_quota_enforcement flag for the actor) with usage at/over the limit
    • current_usage - count of tracked contexts within the organization
    • quota_limit / remaining_quota / quota_explicitly_set? / quota_enforcement_enabled?
    • quota_exceeded_error_message and a quota_info hash for API/UI consumption
  2. New scope for_root_namespaces on Security::ProjectTrackedContext.

Usage counting and the cross-database constraint

Security::ProjectTrackedContext lives on the sec database while Organizations::Organization lives on main, so we cannot join across databases. Usage is counted by scoping tracked contexts to the organization's root namespaces via the denormalized traversal_ids column.

Each root namespace is matched with a half-open range on the whole traversal_ids array (>= '{id}' AND < '{id + 1}'), mirroring Namespaces::Traversal::Traversable#within (the same pattern used by vulnerability_reads). This uses the existing index_security_project_tracked_contexts_on_traversal_ids_id btree index as a range scan, rather than an unindexable traversal_ids[1] element-subscript predicate.

Behaviour

  • Enforcement is gated by the dedicated :security_tracked_context_quota_enforcement feature flag, independently of VAC. The actor follows the rollout granularity (callers pass the project during closed beta). This lets us enable VAC tracking without enforcing the quota, and disable enforcement in an emergency without turning off VAC.
  • A nil quota limit (e.g. self-managed, no SaaS-wide default) means no enforcement
  • The quota is organization-level, counted across all projects in the organization

Follow-up: enforcement is intentionally decoupled from VAC while it is validated in isolation. Once proven, we will bind it into the VAC rollout so VAC-enabled organizations get quota enforcement together.

Database

The only query introduced is the current_usage COUNT. It uses the existing index_security_project_tracked_contexts_on_traversal_ids_id btree (traversal_ids, id) via a range scan.

Query

SELECT COUNT(*)
FROM security_project_tracked_contexts
WHERE state = 2
  AND traversal_ids >= '{9970}' AND traversal_ids < '{9971}';

(For an organization with multiple root namespaces, one range is OR'd per root namespace.)

Query plan (postgres.ai, gitlab-production-sec)

Index Scan, ~71 ms:

Aggregate  (cost=3.59..3.60 rows=1 width=8) (actual time=70.330..70.331 rows=1 loops=1)
  Buffers: shared hit=150 read=545 dirtied=1
  ->  Index Scan using index_security_project_tracked_contexts_on_traversal_ids_id on public.security_project_tracked_contexts  (cost=0.56..3.58 rows=1 width=0) (actual time=2.795..70.248 rows=684 loops=1)
        Index Cond: ((traversal_ids >= '{9970}'::bigint[]) AND (traversal_ids < '{9971}'::bigint[]))
        Filter: (state = 2)
        Rows Removed by Filter: 0
        Buffers: shared hit=150 read=545 dirtied=1
Time: 71.284 ms (planning: 0.900 ms, execution: 70.384 ms)
For comparison: the original `traversal_ids[1] IN (...)` approach (rejected)

An earlier version filtered on traversal_ids[1], an element-subscript predicate that cannot use the btree index, forcing a parallel sequential scan (~3 s, ~5.7M rows filtered, ~1.3 GiB read):

Aggregate  (cost=802832.51..802832.52 rows=1 width=8) (actual time=3035.550..3038.069 rows=1 loops=1)
  ->  Parallel Seq Scan on public.security_project_tracked_contexts  (cost=0.00..801742.96 rows=35732 width=0) (actual time=35.247..3031.018 rows=228 loops=3)
        Filter: ((state = 2) AND (traversal_ids[1] = 9970))
        Rows Removed by Filter: 5705472
Time: 3.039 s

The range-based rewrite avoids this entirely, so no new index or column is required.

How to set up and validate locally

org = Organizations::Organization.first
service = Security::TrackedContextQuotaService.new(org)

service.quota_info
# => { limit:, usage:, remaining:, explicitly_set: }

# Pass an actor (e.g. a project) so enforcement resolves the
# :security_tracked_context_quota_enforcement flag for that actor.
service.quota_available?(project)

Run the specs:

bundle exec rspec ee/spec/services/security/tracked_context_quota_service_spec.rb

MR acceptance checklist

This MR adds a service that is not yet wired into any caller. The validation that consumes it lands in #598088.

Closes #598087

Part of &20475 (Tracked Context Quota Management)

Edited by Gregory Havenga

Merge request reports

Loading
Loading