Skip to content

Added aggregation lease column in application settings

What does this MR do and why?

We have made another attempt to reduce the lease duration for Namespace::AggregationSchedule. After several iterations, we have managed to reduce the lease duration to 2 minutes, with the feature flag enabled for gitlab-org on production. However, we have decided to wait back on the rollout of the feature flag with a 2-minute lease duration due to monitoring the rollout. Based on the analysis in this thread, we have realized that the ContainerRegistryAPI cannot handle the increased load that might result from fully rolling out the feature flag with a 2-minute lease duration. Therefore, we have decided to increase the lease duration to 5 minutes and monitor the situation. We have also decided to do this via application_setting, which will allow us to make adjustments to the lease duration if the load results in uncertain situations with the ContainerRegistry API.

  1. This MR just adds the ApplicationSetting
  2. The next MR will use this ApplicationSetting to change the lease duration from 2 mins to 5 mins

ref: https://gitlab.com/gitlab-org/gitlab/-/issues/370251

Added aggregation lease column in application settings

Changelog: added

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Database

up

main: == [advisory_lock_connection] object_id: 275880, pg_backend_pid: 63146
main: == 20230517090826 AddNamespaceAggregationScheduleLeaseDurationToApplicationSettings: migrating
main: -- add_column(:application_settings, :namespace_aggregation_schedule_lease_duration_in_seconds, :integer, {:default=>5 minutes, :null=>false})
main:    -> 0.0052s
main: == 20230517090826 AddNamespaceAggregationScheduleLeaseDurationToApplicationSettings: migrated (0.0086s)

main: == [advisory_lock_connection] object_id: 275880, pg_backend_pid: 63146
ci: == [advisory_lock_connection] object_id: 276140, pg_backend_pid: 63148
ci: == 20230517090826 AddNamespaceAggregationScheduleLeaseDurationToApplicationSettings: migrating
ci: -- add_column(:application_settings, :namespace_aggregation_schedule_lease_duration_in_seconds, :integer, {:default=>5 minutes, :null=>false})
ci:    -> 0.0049s
ci: == 20230517090826 AddNamespaceAggregationScheduleLeaseDurationToApplicationSettings: migrated (0.0127s)

ci: == [advisory_lock_connection] object_id: 276140, pg_backend_pid: 63148

down

main: == [advisory_lock_connection] object_id: 275700, pg_backend_pid: 21508
main: == 20230517090826 AddNamespaceAggregationScheduleLeaseDurationToApplicationSettings: reverting
main: -- remove_column(:application_settings, :namespace_aggregation_schedule_lease_duration_in_seconds, :integer, {:default=>5 minutes, :null=>false})
main:    -> 0.0021s
main: == 20230517090826 AddNamespaceAggregationScheduleLeaseDurationToApplicationSettings: reverted (0.0063s)

main: == [advisory_lock_connection] object_id: 275700, pg_backend_pid: 21508


ci: == [advisory_lock_connection] object_id: 275640, pg_backend_pid: 21959
ci: == 20230517090826 AddNamespaceAggregationScheduleLeaseDurationToApplicationSettings: reverting
ci: -- remove_column(:application_settings, :namespace_aggregation_schedule_lease_duration_in_seconds, :integer, {:default=>5 minutes, :null=>false})
ci:    -> 0.0028s
ci: == 20230517090826 AddNamespaceAggregationScheduleLeaseDurationToApplicationSettings: reverted (0.0121s)

ci: == [advisory_lock_connection] object_id: 275640, pg_backend_pid: 21959

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Suraj Tripathi

Merge request reports