Skip to content

Extract service ping payload persistance outside of SubmitService

What does this MR do and why?

This is a follow up to !84886 (merged) that continues on implementation of #354139 (comment 892812096)

This MR extract ServicePing payload computation and persistance out of SubmitService into GitlabServicePingWorker in order to pre compute SerivcePing data for preview for instance admins even if those instances does not submit this data later on to GitLab

It also makes change to how permitted data categories are resolved. Prior to this MR when ServicePing was disabled all data categories was considered as not permitted. With this MR all data categories will be considered as permitted when ServicePing is disabled This change was made in order to generate complete preview ServicePing data for instance administrators.

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Before each scenario:

  1. Spin up local VersionApp server. If you do not have VersionApp set up locally follow this instruction

  2. Change ServicePing:SubmitService to point to your local development VersionApp instance

diff --git a/app/services/service_ping/submit_service.rb b/app/services/service_ping/submit_service.rb
index 64793f5072c..e8236f9142d 100644
--- a/app/services/service_ping/submit_service.rb
+++ b/app/services/service_ping/submit_service.rb
@@ -102,6 +102,7 @@ def save_raw_usage_data(usage_data)
     # See https://gitlab.com/gitlab-org/gitlab/-/issues/233615 for details
     def base_url
       Rails.env.production? ? PRODUCTION_BASE_URL : STAGING_BASE_URL
+      'http://localhost:3000'
     end
   end
 end

Scenario A (ServicePing reporting is disabled)

In rails console bundle exec rails c check current state and trigger process

[1] pry(main)> ApplicationSetting.current.update(usage_ping_enabled: false)
[2] pry(main)> RawUsageData.count
   (0.4ms)  SELECT COUNT(*) FROM "raw_usage_data" /*application:console,db_config_name:main,line:(pry):6:in `__pry__'*/
=> 0
[3] pry(main)> GitlabServicePingWorker.new.perform

Confirm success by:

  1. New RawUsageData record was created
    [4] pry(main)> RawUsageData.count
      (0.4ms)  SELECT COUNT(*) FROM "raw_usage_data" /*application:console,db_config_name:main,line:(pry):3:in `__pry__'*/
    => 1
  2. No http request was sent to VersionApp

Scenario B (ServicePing reporting is enabled)

In rails console bundle exec rails c

[1] pry(main)> ApplicationSetting.current.update(usage_ping_enabled: true)
[2] pry(main)> RawUsageData.count
   (0.4ms)  SELECT COUNT(*) FROM "raw_usage_data" /*application:console,db_config_name:main,line:(pry):6:in `__pry__'*/
=> 0
[3] pry(main)> GitlabServicePingWorker.new.perform

Confirm success by:

  1. New RawUsageData record was created

    [4] pry(main)> RawUsageData.count
      (0.4ms)  SELECT COUNT(*) FROM "raw_usage_data" /*application:console,db_config_name:main,line:(pry):3:in `__pry__'*/
    => 1
  2. In VersionApp 2 http requests were received

    Started POST "/usage_data" for ::1 at 2022-05-05 14:32:38 +0200
    (1.6ms)  SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
    Processing by UsageDataController#create as HTML
    Parameters: {"recorded_at"=>"[FILTERED]", "uuid"=>"6b3578df-146d-4915-af4a-00fda80abac8",   
    ...
    Completed 201 Created in 3666ms (Views: 0.3ms | ActiveRecord: 115.6ms | Allocations: 599620)
    
    Started POST "/usage_ping_metadata" for ::1 at 2022-05-05 14:32:42 +0200
    Processing by UsagePingMetadataController#create as HTML
    Parameters: {"metadata"=>{"metrics"=>[{"name"=>"version", "time_elapsed"=>2.9999937396496534e-06}, ....   {"name"=>"counts.ci_builds", "time_elapsed"=>0.017847000010078773}, {"name"=>"counts.ci_internal_pipelines", "time_elapsed"=>0.016969999996945262}]
    Completed 201 Created in 18ms (ActiveRecord: 0.0ms | Allocations: 37556)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Mikołaj Wawrzyniak

Merge request reports