Increase disk size for saas-linux-* Hosted Runners for GitLab.com
We've recently updated the base image used by ephemeral VMs executing jobs for our saas-linux-*
runners. However, the new image uses a little more of the available disk space than its predecessor did.
This means that some jobs that were properly working on our Hosted Runners on Linux for GitLab.com until tomorrow - but were at the very edge of available disk space - now may start failing only because we've changed the OS environment.
I think we should consider increasing the disk size of the saas-linux-small-amd64
runners.
These have currently the smallest disk space available:
$ grep disk-size roles/runners-manager-saas-linux-*
roles/runners-manager-saas-linux-2xlarge-amd64.json: "google-disk-size": 200,
roles/runners-manager-saas-linux-large-amd64.json: "google-disk-size": 100,
roles/runners-manager-saas-linux-large-arm64.json: "google-disk-size": 100,
roles/runners-manager-saas-linux-medium-amd64-gpu-standard.json: "google-disk-size": 50,
roles/runners-manager-saas-linux-medium-amd64.json: "google-disk-size": 50,
roles/runners-manager-saas-linux-medium-arm64.json: "google-disk-size": 50,
roles/runners-manager-saas-linux-small-amd64.json: "google-disk-size": 25,
roles/runners-manager-saas-linux-xlarge-amd64.json: "google-disk-size": 200,
While we probably don't need 200 GB nor even 100 GB here - this is the benefit of the larger, premium runners, 25GB is just too little. We've found that on a freshly created VM without any usage generated on it yet, the available disk space was about 16 GB before yesterday's change, and now it's about 14 GB:
tomasz@tm-test-cos-space-before ~ $ sudo df -h /var/lib/docker
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 21G 2.6G 18G 13% /var/lib/docker
tomasz@tm-test-cos-space-after ~ $ sudo df -h /var/lib/docker
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 21G 4.5G 16G 22% /var/lib/docker
I propose to align the disk size with the medium runners (50 GB) or increase it to at least 30 GB, which will compensate the additional disk consumption generated by the updated base OS.