Add observability for package metadata bucket-labels sync path

Summary

Add Rails-side observability to the package metadata bucket-labels sync path introduced in !230816 (merged) / !235896 (merged). We need to measure adoption of the labels path vs the listObjects fallback so we can confirm the cost win and catch regressions.

Background

#590584 (closed) shipped reading GCS bucket labels first to skip expensive listObject calls. The feature is default-enabled in 19.0. Today we only have debug logs and no counters, so we can't answer "how often is each instance hitting the labels path?".

Internal event tracking

Use internal events, same pattern as cvs_on_sbom_change (!163935 (merged)), track_internal_event called from the connector with additional_properties.

Event definitions:

ee/config/events/package_metadata_sync_v2_advisories.yml:

description: Tracks which sync path was resolved per data_after call in GCS connector for advisories
internal_events: true
action: package_metadata_sync_v2_advisories
additional_properties:
  label:
    description: purl_type (e.g. npm, pypi, gem)
  property:
    description: "sync path taken (see table below)"
product_group: composition_analysis
product_categories:
  - software_composition_analysis
milestone: '19.1'
tiers:
  - ultimate

ee/config/events/package_metadata_sync_v2_licenses.yml:

description: Tracks which sync path was resolved per data_after call in GCS connector for licenses
internal_events: true
action: package_metadata_sync_v2_licenses
additional_properties:
  label:
    description: purl_type (e.g. npm, pypi, gem)
  property:
    description: "sync path taken (see table below)"
product_group: composition_analysis
product_categories:
  - software_composition_analysis
milestone: '19.1'
tiers:
  - ultimate

Instrumentation in Gcp#data_after and #retrieve_data_from_labels

include Gitlab::InternalEventsTracking

# for advisories
track_internal_event(
  'package_metadata_sync_v2_advisories',
  additional_properties: {
    property: 'labels_hit',
    label: sync_config.purl_type
  }
)

# for licenses
track_internal_event(
  'package_metadata_sync_v2_licenses',
  additional_properties: {
    property: 'labels_hit',
    label: sync_config.purl_type
  }
)

Paths tracked:

property branch
listobjects_blank_checkpoint checkpoint.blank?
listobjects_v1 !sync_config.v2?
listobjects_cve_enrichment sync_config.cve_enrichment?
labels_fallback_no_labels entries.empty?
labels_caught_up caught_up?
labels_fallback_stale_checkpoint data_missing_from_labels?
labels_hit labels path success
labels_fallback_gcs_error rescue Google::Cloud::Error

Related

Edited by Ahmad Zaydan