Convert shared_runners_duration to bigint for project and namespace monthly usage tables
This issue is to convert the type of shared_runners_duration
from integer
to bigint
for both table ci_project_monthly_usages
and ci_namespace_monthly_usages
as soon as we can by the end of 2023-08-31
. Otherwise, we will encounter the same problem by the end of each month (see below).
Why are we doing this work
A production spike has appeared and an issue #420342 (closed) is logged by the bot.
This is one of the problems we need to resolve: the shared_runners_duration
is out of the range of type integer
Click to expand the investigation
The monthly usage for namespace #<Group @gitlab-org>
already has a duration 2147483647
and the build duration to add is 271.414939
, therefore, the new duration amount 2147483647.414939
exceeds the range (-2147483648
to +2147483647
) of Postgres type integer
for column shared_runners_duration
in table ci_namespace_monthly_usages
. To resolve this, we will need to increase the range of the type to e.g. bigint
.
[ gprd ] production> usage1 = Ci::Minutes::NamespaceMonthlyUsage.where(date: '2023-07-01').wh
ere(namespace_id: 9970).first
=>
#<Ci::Minutes::NamespaceMonthlyUsage:0x00007f291bfdc528
...
[ gprd ] production> usage1.attributes
=>
{"id"=>14544619,
"namespace_id"=>9970,
"date"=>Sat, 01 Jul 2023,
"notification_level"=>100,
"shared_runners_duration"=>2147483647,
"created_at"=>Sat, 01 Jul 2023 00:00:10.186695000 UTC +00:00,
"amount_used"=>0.0}
Therefore, in this issue, we will need to convert the type of shared_runners_duration
to bigint
for both table ci_project_monthly_usages
and ci_namespace_monthly_usages
.
Click to expand why it's needed for ci_project_monthly_usages
Considering the increasing usage for project #<Project gitlab-org/gitlab>
is already 1766708085
for month 2023-07-01
, we will need to do the conversion for ci_project_monthly_usages
:
[ gprd ] production> usage = Ci::Minutes::ProjectMonthlyUsage.where(date: '2023-07-01').where
(project_id: 278964).first
=>
#<Ci::Minutes::ProjectMonthlyUsage:0x00007f2903586910
...
[ gprd ] production> usage.attributes
=>
{"id"=>6585962,
"project_id"=>278964,
"date"=>Sat, 01 Jul 2023,
"shared_runners_duration"=>1766708085,
"created_at"=>Sat, 01 Jul 2023 00:00:09.675314000 UTC +00:00,
"amount_used"=>0.0}
Please follow the instruction to convert a column to bigint at https://docs.gitlab.com/ee/development/database/avoiding_downtime_in_migrations.html#changing-column-types or https://docs.gitlab.com/ee/development/database/avoiding_downtime_in_migrations.html#migrating-integer-primary-keys-to-bigint depending on how big the tables are and how long the migration will take
Relevant links
Non-functional requirements
-
Documentation: -
Feature flag: -
Performance: -
Testing: