Resolve "Backend: Add Service Ping metrics for AI Catalog"

What does this MR do and why?

This MR adds metric definitions to track different numbers across AI catalog items and their consumers. This MR adds 6 different metrics - see the issue #558487 (closed) description for details.

How to set up and validate locally

  1. Checkout branch 558487-add-service-ping-metrics-for-ai-catalog
  2. Open terminal and setup Internal Events Monitor as per the guidance
  3. Now create AI catalog items (agent/flow) and it's consumer.
  4. After creating these records you can see it on you terminal through Internal Events Monitor.

Execute below from rails console


Feature.enable(:global_ai_catalog) # Enable the global_ai_catalog feature flag:

# create ai catalog items
public_agent = FactoryBot.create(:ai_catalog_agent, project_id: 1000000, public: true)
private_agent = FactoryBot.create(:ai_catalog_agent, project_id: 1000000, public: false)
public_flow = FactoryBot.create(:ai_catalog_flow, project_id: 1000000, public: true)
private_flow = FactoryBot.create(:ai_catalog_flow, project_id: 1000000, public: false)


# create consumers
FactoryBot.create(:ai_catalog_item_consumer, organization: public_flow.organization, item: public_flow, enabled: true)
FactoryBot.create(:ai_catalog_item_consumer, organization: public_flow.organization, item: private_flow, enabled: false)

Monitor metrics from terminal


# execute below on terminal from gitlab project

rails runner scripts/internal_events/monitor.rb counts.ai_catalog_public_flows counts.ai_catalog_private_flows counts.ai_catalog_public_agents counts.ai_catalog_private_agents counts.ai_catalog_flows_enabled_across_groups counts.ai_catalog_flows_enabled_across_projects

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #558487 (closed) This code adds tracking and measurement capabilities for GitLab's AI Catalog feature. It creates six new metrics to count different types of AI catalog items:

  1. AI flows enabled across projects and groups - tracks how many AI workflows are actively being used by teams
  2. Public and private AI flows - counts AI workflows that are either shared publicly or kept private
  3. Public and private AI agents - counts AI assistants that are either shared publicly or kept private

The code includes two main measurement tools:

  • One that counts enabled AI flows at the project and group level
  • Another that counts different types of AI catalog items (flows and agents) based on their visibility (public/private)

All metrics are configured to work only with premium and ultimate GitLab tiers and will be available starting in version 18.4. The code also includes comprehensive tests to ensure the counting works correctly under various scenarios, including when no items exist or when items are disabled.

This helps GitLab understand how customers are adopting and using their AI catalog features across different organizational levels.

Edited by Jaydip Pansuriya

Merge request reports

Loading