Draft: Create the default tracked context untracked when over quota

Gregory Havengarequested to merge
ghavenga-621576-untracked-default-over-quota into ghavenga-20475-root-namespace-quota

What does this MR do and why?

With the security_tracked_context_quota_enforcement feature flag on, the default branch context of a project counts toward the tracked context quota. Before this MR, a project could not get a default context when its root namespace was at the quota. The create failed validation with the quota error, and security report ingestion raised an error on each default branch pipeline.

This MR creates the default context untracked in that case. GitLab stores the pipeline findings but does not ingest them. Users can track the default branch again when the quota has space.

Stacked MR. This MR targets ghavenga-20475-root-namespace-quota (!265368), which adds the root namespace quota and Security::TrackedContextQuota. It retargets to master when that MR merges.

Changes

  • FindOrCreateService creates a new default context in the untracked state when the quota is not available (Security::TrackedContextQuota.available_for?).
  • Security::ProjectTrackedContext#default_ref_cannot_be_untracked accepts a new untracked default context when the quota is not available. It also accepts a saved untracked default context that does not change state, for example a rename. It does not let an existing tracked default context become untracked.
  • IngestReportsService returns early for an untracked default context. It writes an info log with the project ID, the pipeline ID and the tracked context ID. It does not raise. It still schedules the approval rule sync, because the security findings are stored. The other steps do not run for that pipeline, which includes vulnerability ingestion, SBOM ingestion, mark as resolved and the latest pipeline update.
  • StoreScansService#ensure_default_tracked_context does not try to create the default context when an untracked default context exists. Without this check, it logs a failure on every default branch pipeline.
  • CreateService (used by the securityRefsTrack mutation) tracks an existing untracked context instead of failing on the duplicate name. This applies to any untracked context, also a non-default context that a pipeline security tab action created. The context keeps its is_default value. The quota validation runs on that transition, so the request fails with the quota error when the quota is still reached.
  • Report#tracked_context returns the untracked default context when no tracked context exists for the pipeline. Security findings then keep context-aware (v2) UUIDs while the default context is untracked. Vulnerabilities that users create or dismiss from the pipeline security tab use the same UUIDs as later ingestion, so no duplicates appear after the default branch is tracked again.
  • The error response for an existing untracked context from FindOrCreateService has the reason :context_not_tracked and the context in the payload. The message does not change.

Behavior

Feature flag Quota Default context on create Ingestion Store phase UUIDs
Off Any Tracked Runs Context-aware
On Not reached Tracked Runs Context-aware
On Reached Untracked Skipped, info log Context-aware

Users track the default branch again with securityRefsTrack. The next default branch pipeline then ingests the reports. A follow-up MR adds the state and a Track action to the ref list on the security configuration page. Whether GitLab tracks these default branches again automatically is an open product question on #621576.

If we disable the feature flag, the untracked default contexts stay untracked. Ingestion continues to skip them until a user tracks them again.

How to set up and validate locally

  1. Enable the feature flags. Set a quota of 1 on the root namespace of a project. Another project in the same root namespace must already have 1 tracked context.

    Feature.enable(:vulnerabilities_across_contexts, project)
Feature.enable(:security_tracked_context_quota_enforcement, project)
project.root_namespace.namespace_settings.update!(security_tracked_context_quota: 1)

  2. Make sure that the project has no default branch context. Then run:

    Security::ProjectTrackedContexts::FindOrCreateService.project_default_branch(project).execute

  3. Check that the result is a success and that the context is untracked.

  4. Run a pipeline with a security report on the default branch. Check that ingestion does not raise and that the log contains Skipped ingesting security reports for untracked default context.

  5. Raise the quota to 10:

    project.root_namespace.namespace_settings.update!(security_tracked_context_quota: 10)

  6. Track the default branch with the securityRefsTrack mutation. Check that the context is now tracked and that is_default is still true.

    mutation {
  securityRefsTrack(input: { projectPath: "<project full path>", refs: [{ name: "<default branch>", refType: BRANCH }] }) {
    trackedRefs { name }
    errors
  }
}

Specs

  • ee/spec/models/security/project_tracked_context_spec.rb
  • ee/spec/services/security/project_tracked_contexts/find_or_create_service_spec.rb
  • ee/spec/services/security/project_tracked_contexts/create_service_spec.rb
  • ee/spec/services/security/ingestion/ingest_reports_service_spec.rb
  • ee/spec/services/security/store_scans_service_spec.rb
  • ee/spec/lib/ee/gitlab/ci/reports/security/report_spec.rb
  • ee/spec/services/security/vulnerability_uuid_spec.rb

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

Related to #621576. The issue stays open for the question about automatic tracking and the follow-up UI change.

Part of &20475

Merge request reports