Backend: Recalculate remaining minutes on Additional Packs on a monthly basis
Problem
With &5715 we now store any CI/CD minutes pack purchases as Ci::Minutes::AdditionalPack
record. Each pack has an amount, purchase ID and an expiration date.
Currently these additional packs are populated on Gitlab SaaS but there is no logic that on a monthly basis recalculates the remaining minutes.
The existing logic that recalculates remaining minutes operates on namespaces.extra_shared_runners_minutes_limit
column which is where the total number of additional minutes is stored today.
We have this data saved in 2 places:
-
namespaces.extra_shared_runners_minutes_limit
is the legacy column where CustomersDot portal accumulates new purchases. On a monthly basis we reset the CI/CD minutes consumption and also recalculate what is the remaining amount of purchased minutes, if any used. -
Ci::Minutes::AdditionalPack
records is a side-by-side storage of CI/CD minutes purchased. On a monthly basis we DO NOT reset these yet. This is what's being asked in this issue.
Severity
This technical debt is set to severity2 because it represents a feature (AdditionalPacks
) that is not yet been used. The customers, when purchasing additional minutes on the CustomersDot portal, are being told that the pack of additional minutes being purchased expires by X date, usually 1 year after purchase. Purchases of additional minutes increment the extra_shared_runners_minutes_limit
(still used today) and create AdditionaPacks
records, with expiration date, which are not used by the Rails application.
Soon (as purchased made with expiry date start to expire) this behavior will turn into a typebug (from a business perspective) because additional minutes that are supposed to expire, they won't.
Solution
To recalculate the amount in Ci::Minutes::AdditionalPack
s we need to:
- sort non expired additional packs by expiration date and take the ones expiring sooner first.
- If any additional minutes have been used (total usage > monthly limit), reduce the amount from the first additional pack
- iterate with the following additional packs for the remaining amount.
Example:
Customer has used 10,000
minutes. Their monthly limit is 400
. They have purchased the following packs:
- Pack 1, amount
5,000
minutes, expiringJanuary 2022
(expired) - Pack 2, amount
5,000
minutes, expiringMarch 2022
- Pack 3, amount
5,000
minutes, expiringApril 2022
- Pack 4, amount
5,000
minutes, expiringJune 2022
Given that the customer used 9,600
minutes more than their monthly limit (10,000 - 400
), we need to reduce this amount from the additional packs:
- Pack 1 remains as is because it's expired. The customer didn't use the minutes so they are lost.
- Pack 2 becomes
0
(we still have9600 - 5000 = 4600
minutes to reduce) - Pack 3 becomes
400
(5000 - 4600
) - Pack 4 remains as is at
5,000
.
Dependencies and rollout
Given that we already have additional packs on GitLab SaaS, the extra_shared_runners_minutes_limit
does not match anymore with the actual amount in non expired additional packs. To make sure that legacy and new tracking of additional minutes remains consistent we need to follow the plan below:
- Disable temporarily the logic that creates additional packs from CustomersDot to GitLab SaaS.
- Remove existing packs on GitLab SaaS.
-
grouppipeline execution Implement logic behind a disabled feature flag that recalculates the remaining minutes in additional packs and updates
extra_shared_runners_minutes_limit
for the same amount. - For each namespace with a non-zero
extra_shared_runners_minute_limit
, create an additional pack for that amount (without purchase XID, and with an arbitrary expiry date). This would ensure thatextra_shared_runners_minute_limit
and additional packs are in sync. - grouppipeline execution Enable the feature flag above since we now have additional packs.
-
grouppipeline execution Change
Ci::Minutes::Limit
to read purchased minutes as the sum of non-expired packs. This is important since a pack can expire at any time. This can be also done with a feature flag. - Enable the logic that creates additional packs from CustomersDots to GitLab SaaS.
- Deprecate and remove
extra_shared_runners_minute_limit
. - Remove any technical debt code introduced to fix cross-database modifications: #351847 (closed)
References
- A lot of the conversation that sparked this issue is here: #349045 (comment 791454260)