Skip to content
GitLab
Next
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • gitlab-runner gitlab-runner
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 2,506
    • Issues 2,506
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 122
    • Merge requests 122
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.orgGitLab.org
  • gitlab-runnergitlab-runner
  • Issues
  • #5069
Closed
Open
Issue created Dec 06, 2019 by Tomasz Maczukin@tmaczukinMaintainer

Make docker+machine autoscaling configuration more elastic

Currently the autoscaling configuration of docker+machine executor is simple and allows to define two kinds of settings: for normal and OffPeak periods. It may look for example like that:

[docker.machine]
  (...)
  MaxBuilds = 1
  IdleCount = 100
  IdleTime = 3600
  OffPeakPeriods = ["* * * * * sat,sun *"]
  OffPeakTimezone = "UTC"
  OffPeakIdleCount = 70

The configuration defines that normally we should have 100 machines in idle state, except for whole Saturdays and Sundays, when it should have only 10 idle machines.

While this works well, we've found that a possibility to define more custom settings for different timings is needed. The above example is taken from GitLab.com Shared Runners config. We want to have a little less idle machines at weekends, when the load is smaller. However while working on resolving a 4am spikes problem, we'd decide that we want to define specific settings for around the 4am hour each day apart of existing configuration.

The proposal is to create new configuration syntax:

[docker.machine]
  (...)
  MaxBuilds = 1

  [[docker.machine.autoscaling]]
    Periods = ["* * * * * * *"] # Maybe use it as default, so for the default cases it could be left undefined (less work to do)?
    Timezone = "UTC"
    IdleCount = 100
    IdleTime = 3600

  [[docker.machine.autoscaling]]
    Periods = ["* * * * * sat,sun *"] # The weekends, configured above with the 'OffPeak' settings
    Timezone = "UTC"
    IdleCount = 70
    IdleTime = 3600

  [[docker.machine.autoscaling]]
    Periods = ["* * 3-4 * * * *"] # The 'around 4am' case
    Timezone = "UTC"
    IdleCount = 1000
    IdleTime = 3600

In this case Runner would iterate over all docker.machine.autoscaling definitions searching for the last one that matches current time. And then it would apply the configuration values. So looking on the above example, normally we would have 100 idle machines, at weekends it would be reduced to 70, and at any day (no matter if it's a weekend or not) at 3-4am UTC it would be increased to 1000.

While implementing this we should also deprecate the old configuration values, since these would allow to get the same result and something more. When the old configuration only would be used, we should log then a deprecation notice. If both would be used - we should log that the old one is disabled (and repeat the deprecation notice).

We should also consider if we want to require at least one docker.machine.autoscaling entry with explicit configuration, or do we don't care and just default to 0 for IdleCount and IdleTime values. This would be the easiest approach and would also resolve the case of a configuration that doesn't cover some specific time period.

Edited Mar 31, 2020 by Lyubomir Raykov
Assignee
Assign to
Time tracking