Automate the removal of inactive runners from GitLab UI
I'm using a maintained AWS Spot Fleet to host my build runners. As spot pricing increases and servers disappear, I end up with runners that no longer exist still attached to projects. I'd like a way to automate removal of dead runners.
Proposal
- Create a
namespace_settings.auto_prune_stale_runners
boolean table field with partial indexON auto_prune_stale_runners = TRUE
; - Create cron-type Sidekiq worker that scans namespaces with
auto_prune_stale_runners
set to true and sends ranges of stale runners from those namespaces for deletion into another worker.
-
This cron-type worker should not be re-entrant, meaning that if it executes daily and one execution takes more than one day, the next runs should be discarded until the current one finishes;
-
We should go through pages of namespaces (e.g. 100 at a time) and scan for stale runners from those namespaces.
SELECT DISTINCT (namespace_id) FROM ci_runner_namespaces JOIN ci_runners ON runner_id = ci_runners.id WHERE greatest (ci_runners.created_at, ci_runners.contacted_at) < (now() - interval '3 months') LIMIT 100 OFFSET 0
Assumptions
- As the initial MVC, only group runners would be considered.
- We want to provide users with a switch to enable automatic removal. If this is not required, the scope becomes considerably smaller.
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.