Skip to content

Count duration of shared runners usage separately from CI minutes for viewing of consumption

Release notes

TBD.

Problem to solve

There has been a lot of confusion by open source projects and contributors about the latest announcement to cut CI minutes for Free users. Public projects' CI minutes are not being counted, and this is being called a bug: #243722 (closed)

To my understanding, we decided to call it a bug for now in order to buy time. We want to consider the way that the announcement affects open source projects and contributors. See: https://gitlab.com/gitlab-com/packaging-and-pricing/pricing-handbook/-/issues/204

Unfortunately, having [effectively] unlimited CI mins for public projects be classified as a bug makes it seem like the bug can be fixed at any time without sufficient warning to users.

In summary, these are the challenges associated with calling this a bug:

  • No visibility on CI minute usage by public projects. CI minutes continue to not be counted for public projects and this means there is no visibility on how many minutes are being used. Projects can't make adjustments easily without this visibility, and we aren't able to inform any policies with actual usage data.
  • Open source contributions are in jeopardy. Some open source projects require that contributors test out their contributions using CI/CD before submitting (see comment from user here). Others like the Samba project, a large and well-known open source project/community, would not be able to have open source contributions made to their project if the new limits apply to them (see comment from user here).

Intended users

The intended users would be, primarily, open source projects hosted on GitLab.com and their contributors.

User experience goal

We would want public project account holders to be able to view their CI minute usage, but not have it count against their limit. It would effectively be "for display only" as @fabiopitino mentions here: #243722 (comment 415924742)

Proposal

From #254231 (comment 638852313):

Today the monthly charts will display the CI minutes consumption (in terms of credits/points) and not in terms of duration, because they are multiplied by the cost factor at the time the jobs completed.

Perhaps we could add a new column in ci_namespace_monthly_usage and ci_project_monthly_usage to accumulate the "duration in minutes" for all projects/namespaces. This value would be reset automatically on a monthly basis and it won't be used to enforce CI minutes consumption since we already have a column for that that takes in consideration the cost factor.

Once we track the duration of all shared runners usage per namespace/project we can make better predictions on limits. E.g. A namespace using on average 1000 minutes would be billed as:

  • up to 1000 CI minutes if it uses only private projects
  • as low as 8 CI minutes if it uses only public projects (current cost factor of 0.008)
  • about 166 CI minutes if it uses only MacOS runners (assuming a cost factor of 6.0)

We need to:

  • add new column ci_namespace_monthly_usages.duration (or better shared_runners_minutes_duration column to differ from #338830)
  • add new column ci_project_monthly_usages.duration (or better shared_runners_minutes_duration column to differ from #338830)
  • track duration side by side to CI minutes consumption
Draft of a solution 
diff --git a/ee/app/models/ci/minutes/namespace_monthly_usage.rb b/ee/app/models/ci/minutes/namespace_monthly_usage.rb
index 064355a4f03..4e55edc38ca 100644
--- a/ee/app/models/ci/minutes/namespace_monthly_usage.rb
+++ b/ee/app/models/ci/minutes/namespace_monthly_usage.rb
@@ -27,13 +27,13 @@ def self.find_or_create_current(namespace_id:)
         current_month.safe_find_or_create_by(namespace_id: namespace_id)
       end
 
-      def self.increase_usage(usage, amount)
+      def self.increase_usage(usage, amount, duration = 0)
         return unless amount > 0
 
         # The use of `update_counters` ensures we do a SQL update rather than
         # incrementing the counter for the object in memory and then save it.
         # This is better for concurrent updates.
-        update_counters(usage, amount_used: amount)
+        update_counters(usage, amount_used: amount, duration: duration)
       end
     end
   end
diff --git a/ee/app/models/ci/minutes/project_monthly_usage.rb b/ee/app/models/ci/minutes/project_monthly_usage.rb
index ad134c36aa4..39d87e11668 100644
--- a/ee/app/models/ci/minutes/project_monthly_usage.rb
+++ b/ee/app/models/ci/minutes/project_monthly_usage.rb
@@ -30,13 +30,13 @@ def self.find_or_create_current(project_id:)
         current_month.safe_find_or_create_by(project_id: project_id)
       end
 
-      def self.increase_usage(usage, amount)
+      def self.increase_usage(usage, amount, duration = 0)
         return unless amount > 0
 
         # The use of `update_counters` ensures we do a SQL update rather than
         # incrementing the counter for the object in memory and then save it.
         # This is better for concurrent updates.
-        update_counters(usage, amount_used: amount)
+        update_counters(usage, amount_used: amount, duration: duration)
       end
     end
   end
diff --git a/ee/app/services/ci/minutes/update_build_minutes_service.rb b/ee/app/services/ci/minutes/update_build_minutes_service.rb
index 9b5463f2f94..b5e123f87f1 100644
--- a/ee/app/services/ci/minutes/update_build_minutes_service.rb
+++ b/ee/app/services/ci/minutes/update_build_minutes_service.rb
@@ -14,17 +14,17 @@ def execute(build)
-         return unless build.cost_factor_enabled?
         return unless build.complete?
         return unless build.duration&.positive?
+        return unless build&.runner&.instance_type?

         consumption = ::Gitlab::Ci::Minutes::BuildConsumption.new(build, build.duration).amount
 
-        return unless consumption > 0
 
-        update_minutes(consumption)
+        update_minutes(consumption, build.duration)
         compare_with_live_consumption(build, consumption)
       end
 
       private
 
-      def update_minutes(consumption)
+      def update_minutes(consumption, duration)
         if ::Feature.enabled?(:cancel_pipelines_prior_to_destroy, project, default_enabled: :yaml)
-          ::Ci::Minutes::UpdateProjectAndNamespaceUsageWorker.perform_async(consumption, project.id, namespace.id)
+          ::Ci::Minutes::UpdateProjectAndNamespaceUsageWorker.perform_async(consumption, project.id, namespace.id, duration)
         else
-          ::Ci::Minutes::UpdateProjectAndNamespaceUsageService.new(project.id, namespace.id).execute(consumption)
+          ::Ci::Minutes::UpdateProjectAndNamespaceUsageService.new(project.id, namespace.id).execute(consumption, duration)
         end
       end
 
diff --git a/ee/app/services/ci/minutes/update_project_and_namespace_usage_service.rb b/ee/app/services/ci/minutes/update_project_and_namespace_usage_service.rb
index 831eb224b41..8b362170760 100644
--- a/ee/app/services/ci/minutes/update_project_and_namespace_usage_service.rb
+++ b/ee/app/services/ci/minutes/update_project_and_namespace_usage_service.rb
@@ -13,10 +13,10 @@ def initialize(project_id, namespace_id)
       end
 
       # Updates the project and namespace usage based on the passed consumption amount
-      def execute(consumption)
+      def execute(consumption, duration = nil)
         legacy_track_usage_of_monthly_minutes(consumption)
         ApplicationRecord.transaction do
-          track_usage_of_monthly_minutes(consumption)
+          track_usage_of_monthly_minutes(consumption, duration)
 
           send_minutes_email_notification
         end
@@ -37,12 +37,12 @@ def legacy_track_usage_of_monthly_minutes(consumption)
         update_legacy_namespace_minutes(consumption_in_seconds)
       end
 
-      def track_usage_of_monthly_minutes(consumption)
+      def track_usage_of_monthly_minutes(consumption, duration)
         # TODO(issue 335885): Remove @project
         return unless Feature.enabled?(:ci_minutes_monthly_tracking, @project, default_enabled: :yaml)
 
-        ::Ci::Minutes::NamespaceMonthlyUsage.increase_usage(namespace_usage, consumption) if namespace_usage
-        ::Ci::Minutes::ProjectMonthlyUsage.increase_usage(project_usage, consumption) if project_usage
+        ::Ci::Minutes::NamespaceMonthlyUsage.increase_usage(namespace_usage, consumption, duration) if namespace_usage
+        ::Ci::Minutes::ProjectMonthlyUsage.increase_usage(project_usage, consumption, duration) if project_usage
       end
 
       def update_legacy_project_minutes(consumption_in_seconds)
diff --git a/ee/app/workers/ci/minutes/update_project_and_namespace_usage_worker.rb b/ee/app/workers/ci/minutes/update_project_and_namespace_usage_worker.rb
index fd97240903f..4f0f8e7a517 100644
--- a/ee/app/workers/ci/minutes/update_project_and_namespace_usage_worker.rb
+++ b/ee/app/workers/ci/minutes/update_project_and_namespace_usage_worker.rb
@@ -9,8 +9,8 @@ class UpdateProjectAndNamespaceUsageWorker # rubocop:disable Scalability/Idempot
       urgency :low
       data_consistency :always # primarily performs writes
 
-      def perform(consumption, project_id, namespace_id)
-        ::Ci::Minutes::UpdateProjectAndNamespaceUsageService.new(project_id, namespace_id).execute(consumption)
+      def perform(consumption, project_id, namespace_id, duration = nil)
+        ::Ci::Minutes::UpdateProjectAndNamespaceUsageService.new(project_id, namespace_id).execute(consumption, duration)
       end
     end
   end

Further details

As written in issue: #243722 (comment 415905370)...

Here are some proposed next steps for how to deal with this bug:

  1. Revisit our policy about public projects and CI/CD minute accumulation in relation to open source contribution. We should make sure that we do not penalize people for contributing to open source projects or hosting their open source projects on GitLab. Instead, we should encourage this activity. We have heard from focus groups and users on our forum that sometimes open source projects require contributors to test out their contributions (run CI), which means some Free users will need to use up their own CI mins.

  2. Fix this bug. It's useful for everyone (including us), to understand how many CI minutes are being used by public projects. People can't make adjustments easily if they don't have visibility into how many CI minutes they are using. When we fix this bug, we should not enforce CI minute limits on public projects for at least 3 months, if at all, in order to allow people and projects to adjust. (Deciding on the amount of time we allow for transition, if relevant, should be part of step 1)

  3. Enforce limits. We should enforce our policy only once step 1 and 2 are complete and there has been sufficient adjustment time permitted.

Summary of analysis to date (revised 2020-10-30):

  • The data sources in Sisense for the cost impact estimates are:
  • How to read the Sisense report:
    • CI MINUTES: The total minutes used per plan per month.
    • OVERAGE MINUTES: The total overage minutes incurred per plan per month.
  • Average monthly CI minutes used by public projects in FY21 to date = 12.8m
  • Average monthly overage minutes used by public projects in FY21 = 11m (so the overage rate is at 85%)
  • Public projects in the free plan account for 82% of the overages.

Permissions and Security

Documentation

Availability & Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Is this a cross-stage feature?

Links / references

Edited by Jackie Porter