Make Cost Factor Admin UI use the new columns for cost factor
Problem
Today we have the subsidized cost factor 0.008
that overrides the runner cost factors public_projects_minutes_cost_factor
or private_projects_minutes_cost_factor
depending on some conditions (if the project exists in a namespace created after July 2021, or if the project is a fork of a public project, etc.).
After we enforce CI minutes for all public projects we should remove this condition in the code and convert the cost factor to be set in the database. We realized that choosing a cost factor based on the project visibility become irrelevant. We need to switch cost factor based on standard
condition (applied to most of the projects) and discounted
condition (subsidized cost factor applied based on some rules: e.g. project is fork of OSS project).
From https://gitlab.com/gitlab-org/gitlab/-/issues/350535#note_828267286:
In order to move towards enforcing CI minutes for all projects incrementally, we added application logic to amend, at runtime, the cost factor coming from the runner. This logic allowed us to enforce a subsidised cost factor
0.008
on newly created public projects while keeping old public projects still unlimited. This was the first step toward reducing the number of projects with unlimited minutes.What we are realizing with this initiative of enforcing CI minutes is that distinguishing cost factors based on public/private visibility is no longer sufficient. Actually, our plan would be to align
public_project_cost_factor
to the existingprivate_project_cost_factor: 1
but having a different treatment for public forks, since we are noticing that without a low cost factor we would impact OSS contributions or we would need to increase CI minutes limits.Basically we want to change the cost factors as:
- Before:
public_project_cost_factor: 0
andprivate_project_cost_factor: 1
- After:
public_fork_cost_factor: 0.008
andrunner_cost_factor: 1
This final configuration would allow us to:
- Have a limit for all projects/namespaces, except those special namespaces like
gitlab-org
where CI minutes limits are disabled.- Set a minimum cost factor 0.008 which we can decide to tweak in the future. No more
0
cost factors.- Make it possible for special runners (e.g. macOS or Windows runners) to specify their cost factors for normal and fork projects. At this stage we could move the application logic that alters the runner cost factor back to the database, as we have it today so each runner can have 2 custom cost factors.
Solution
Step | Issue |
---|---|
Add new columns ci_runners.discounted_cost_factor with DEFAULT 0.008 and ci_runners.standard_cost_factor with DEFAULT 1.0
|
#363215 (closed) |
Add feature flag to switch to the new columns (cost factor, Admin UI, RunnerMatcher, GraphQL) | #363216 |
Enable the feature flag |
|
Cleanup old cost factor columns | #363217 |
Convert ci_runners.public_projects_minutes_cost_factor: 0
and ci_runners.private_projects_minutes_cost_factor: 1
to ci_runners.discounted_cost_factor: 0.008
and ci_runners.standard_cost_factor: 1
respectively. Then if the project is a public fork we apply one cost factor otherwise we default to the standard_cost_factor
.
-
#363215 (closed) - Add new columns
ci_runners.discounted_cost_factor
withDEFAULT 0.008
andci_runners.standard_cost_factor
withDEFAULT 1.0
-
#363216 - Add feature flag to switch to the new columns
- Control selection of new runner cost factors - If Gitlab.com and project meets requirement for discounted cost factor (project is a public fork) use
discounted_cost_factor
, otherwisestandard_cost_factor
. - Control the Admin UI to set cost factors for shared runners - Allow setting new cost factors and hide legacy ones.
- Change
Gitlab::Ci::Matching::RunnerMatcher
to use the new cost factor columns. - Add new cost factor columns to GraphQL types and mutations and deprecate old cost factors.
- Control selection of new runner cost factors - If Gitlab.com and project meets requirement for discounted cost factor (project is a public fork) use
- this issue - Enable the feature flag
-
#363217 - Cleanup old cost factor columns
- Remove
Ci::Minutes::CostFactor#for_visibility
since we will determine cost factor based on a new algorithm. - Remove references to
public_projects_minutes_cost_factor
andprivate_projects_minutes_cost_factor
.
- Remove
Proposal for this issue
Step 3. Enable the feature flag