Skip to content

CI JWT token usage instrumentation with Service Ping

Leaminn Ma requested to merge 388375-jwt-token-instrumentation into master

What does this MR do and why?

This MR adds instrumentation to track the usage of id_tokens: (JWT token) in the CI pipeline. The count of unique users is recorded in the following Service Ping payload key paths:

redis_hll_counters.ci_secrets_management.i_ci_secrets_management_id_tokens_build_created_monthly

redis_hll_counters.ci_secrets_management.i_ci_secrets_management_id_tokens_build_created_weekly

The instrumentation works by tracking the event when a pipeline Build job is created with id_tokens defined.

How to set up and validate locally

  1. First observe that the Service Ping payload now includes i_ci_secrets_management_id_tokens_build_created_* monthly and weekly keys.

Run the following commands in the console:

service_ping_payload = Gitlab::Usage::ServicePingReport.for(output: :non_sql_metrics_values)
service_ping_payload['redis_hll_counters']['ci_secrets_management']
Screenshot Screenshot_2023-01-24_at_11.55.09_AM
  1. Note that any events tracked today will not appear in the current Service Ping payload. This metric is aggregated weekly so the current payload only shows the data that was tracked from weeks that have passed. So we can instead check the RedisHLL counter directly for the current week's value.

Run the following command and note the outputted value (it should be 0 if you haven't run Step 4 yet).

::Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names: 'i_ci_secrets_management_id_tokens_build_created', start_date: Date.today, end_date: Date.today + 7.days, context: '')
Screenshot Screenshot_2023-01-24_at_12.05.41_PM
  1. Go to your Project's CI/CD Editor and update the contents with:
job_with_id_tokens:
  id_tokens:
    ID_TOKEN_1:
      aud: https://gitlab.com

  script:
    - echo 'test'
  1. Commit the changes and wait for the pipeline to complete running. Now observe the counter value again. It should have incremented by 1.
::Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names: 'i_ci_secrets_management_id_tokens_build_created', start_date: Date.today, end_date: Date.today + 7.days, context: '')
Screenshot Screenshot_2023-01-24_at_12.21.10_PM
  1. (Optional) If you'd like to see the Service Ping payload counters incremented, then you can track a mocked event set to a week ago and then check the Service Ping payload output.
::Gitlab::UsageDataCounters::HLLRedisCounter.track_event('i_ci_secrets_management_id_tokens_build_created', values: 1, time: Date.today - 7.days)
service_ping_payload = Gitlab::Usage::ServicePingReport.for(output: :non_sql_metrics_values)
service_ping_payload['redis_hll_counters']['ci_secrets_management']
Screenshot Screenshot_2023-01-24_at_12.41.36_PM

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #388375 (closed)

Edited by Leaminn Ma

Merge request reports