Set CI/CD limit for projects within a sub-group - self-hosted
Update from Contributor Success Team - 8/17/2022
Progress
- (!91519 (closed)) Rough draft of changes to implement project-specific ci consumption limits set
- Shows one way to implement a default CI consumption limit for projects in a namespace that could then be overridden for individual projects
- Does not address permission requirements
- Additional review commentary
- (!90537 (closed)) Rough draft of changeset for enabling Project-based Ci::Minutes::Usage calculations
- Shows changes in exploring how to implement Project-specific CI/CD limits
- Only modifies calculation and would need significant clean up
- Starts with logic from
EE::Project
module where current namespace-based calculation is and traces throughCi::Minutes
classes - Adds new actor-based calculations
- (!83331 (merged)) Related MR: Minute::Quota renamed to Minute::Usage
Next Steps
-
Get a testable draft ready for testing locally using MR !91519 (closed) as a guideline -
Review testable draft -
Address permission requirements
Problem to solve
As a Self-Managed GitLab Admin, I want to set different minute limits by group (already exists) and sub-group, so that I can maintain granularity of minute usage within my larger instance.
Use Cases
OSS Project, Self Hosted
Using the Drupal project, as an example - we have more than 30,000 separate projects, with new projects automatically created under the same group. However, certain projects (like Drupal core itself, or the most highly used modules) will need higher test limits than others. We don't want to move these projects to new groups, because we want to keep stable git urls (git.drupal.org/project:).
It would also be useful to be able to set limits per sub-group, but per-project is more urgent for our particular use case.
- Drupal has 30,000+ canonical repositories
- They all share the 'project' namespace - because in our transition to GitLab this was what let us preserve the git urls that are already used by the Drupal ecosystem in their automated CI/CD systems and such (e.g: git.drupal.org/project/)
- We also want to use the shared namespace to set consistent default labels, gitlabci.yml templates, etc, etc.
- However, all within that same namespace we have:
- Small contributed modules, that could easily get by with a CI limit of 400 min/month
- Huge, heavily relied-on contributed modules that may need many, many more minutes
- Drupal Core itself, which uses thousands of minutes monthly
With those constraints, our options are fairly limited
- We could try to move some projects into new namespaces
- But if we do this, we have to duplicate our configuration for labels, user roles, gitlabci.yml templates, etc
- AND we would have to find some way to safely re-write the git urls AND avoid collisions - so as not to break thousands (or even 10s of thousands) existing sites using automation based on those stable url patterns.
- We could set a really high CI/CD quota to accommodate Drupal core - but then there is effectively no limit at all on the contrib ecosystem, and that could become expensive/vulnerable to bitcoin miner attacks
- If per-project limits were implemented - we would have the fine grained control we need to give the larger projects enough CI time, and still have cost controls in place.
MVC Proposal
- Add Set limit endpoint to the Projects API that sets minute limit for a project (GraphQL or v4 API tbd)
- Can be set by administrators only. Maximum number of monthly CI/CD minutes for this user. Can be nil (default; inherit system default), 0 (unlimited), or > 0. (this is how it works for the user currently) as noted in the documentation here.
- Note that this should still be within the namespace Quota to respect license quotas for usage of SaaS Runners
- Add Get limit endpoint to the Projects API that provides currently set limit
- To simplify the MVC we will not do a check and stop a job if the pipeline hits the project limit value during a run, only at the start. We will retain the namespace check that happens already though.
Not handled in the MVC
- A way to see minutes remaining from the API
- Any UI for this
- Catching more minutes allocated that the namespace or group has allocated (I have 400 minutes total and have set limit that add up to 500, that is ok).
Proposal Options
- Replicate the group level CI/CD limit configuration option for projects and sub-groups
- Project level limits can be set higher than a sub-group limits or group/namespace quotas but won't be allowed to use that many minutes. Once quota is used up for the time period it is out.
This configuration could also be bulk populated to address issues like #243722 (closed)