Investigate why `update-from-default-branch` and `update-from-mr-branch` jobs fail with "No space left on device" error in CI
Overview
update-from-default-branch job and update-from-mr-branch job have been failing in MR pipelines (e.g. https://gitlab.com/gitlab-org/gitlab-development-kit/-/pipelines/605497889) due to the below error.
--------------------------------------------------------------------------------
Generating gitlab-org/gitlab Rails translations
--------------------------------------------------------------------------------
rake aborted!
Listen::Error::INotifyMaxWatchesExceeded: Unable to monitor directories for changes because iNotify max watches exceeded. See https://github.com/guard/listen/blob/master/README.md#increasing-the-amount-of-inotify-watchers .
make[1]: Leaving directory '/builds/gitlab-org/gitlab-development-kit/gdk'
...
Caused by:
Errno::ENOSPC: No space left on device - Failed to watch "/builds/gitlab-org/gitlab-development-kit/gdk/gitlab/spec/components/previews/pajamas": The user limit on the total number of inotify watches was reached or the kernel failed to allocate a needed resource.
/builds/gitlab-org/gitlab-development-kit/gdk/gitlab/vendor/bundle/ruby/2.7.0/gems/rb-inotify-0.10.1/lib/rb-inotify/watcher.rb:74:in `initialize'
Let's find out why this error occurs and how we can resolve it.
Edited by Ash McKenzie