Gitlab installation fails due to massive memory usage while compile assets

Summary

Test system has 8GB of memory which is the recommended amount: https://gitlab.com/gitlab-org/gitlab-foss/blob/master/doc/install/requirements.md Installation fails due massive memory usage while compiling assets.

Steps to reproduce

I have a vagrant file available to reproduce this problem. You can find it here: https://gitlab.fechner.net/mfechner/Gitlab-vagrant

Make sure you have virtualbox, ansible and vagrant installed on the machine you testing it. Execute the following commands (the disc size of FreeBSD virtualbox image must be increased, so you have to execute vagrant up twice the first time):

git clone https://gitlab.fechner.net/mfechner/Gitlab-vagrant.git
cd Gitlab-vagrant
git checkout current
vagrant up
vagrant destroy
cd ~/.vagrant.d/boxes/freebsd-VAGRANTSLASH-FreeBSD-12.1-RELEASE/2019.11.01/virtualbox
VBoxManage clonehd vagrant.vmdk tmp-disk.vdi --format vdi
VBoxManage modifyhd tmp-disk.vdi --resize 12000
VBoxManage clonehd tmp-disk.vdi resized-disk.vmdk --format vmdk
rm -f tmp-disk.vdi vagrant.vmdk
mv resized-disk.vmdk vagrant.vmdk
cd -
vagrant up

While the asset compilation is running, you can login with vagrant ssh into the machine and use top to see memory consumption. After the memory is full, the kernel starts to kill processes, you can see this in /var/log/messages:

May  6 08:47:33 freebsd kernel: swap_pager: out of swap space
May  6 08:47:33 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:47:35 freebsd kernel: pid 4423 (ruby26), jid 0, uid 211, was killed: out of swap space
May  6 08:47:43 freebsd kernel: pid 3789 (postgres), jid 0, uid 770, was killed: out of swap space
May  6 08:47:43 freebsd kernel: pid 3791 (postgres), jid 0, uid 770, was killed: out of swap space
May  6 08:47:43 freebsd kernel: pid 3786 (postgres), jid 0, uid 770, was killed: out of swap space
May  6 08:47:43 freebsd kernel: pid 3788 (postgres), jid 0, uid 770, was killed: out of swap space
May  6 08:47:43 freebsd kernel: pid 3787 (postgres), jid 0, uid 770, was killed: out of swap space

Example Project

https://gitlab.fechner.net/mfechner/Gitlab-vagrant.git

What is the current bug behavior?

The compilation of assets fails so, gitlab cannot be used.

What is the expected correct behavior?

The compilation of assets should run through without any issues, the command that is used for this, matches the documentation here: https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/doc/install/installation.md#compile-assets

The command which is used for this is: su -l git -c "cd /usr/local/www/gitlab-ce && bundle exec rake gitlab:assets:compile RAILS_ENV=production NODE_ENV=production NODE_OPTIONS='--max_old_space_size=4096'"

Relevant logs and/or screenshots

May  6 08:47:33 freebsd kernel: swap_pager: out of swap space
May  6 08:47:33 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:47:35 freebsd kernel: pid 4423 (ruby26), jid 0, uid 211, was killed: out of swap space
May  6 08:47:43 freebsd kernel: pid 3789 (postgres), jid 0, uid 770, was killed: out of swap space
May  6 08:47:43 freebsd kernel: pid 3791 (postgres), jid 0, uid 770, was killed: out of swap space
May  6 08:47:43 freebsd kernel: pid 3786 (postgres), jid 0, uid 770, was killed: out of swap space
May  6 08:47:43 freebsd kernel: pid 3788 (postgres), jid 0, uid 770, was killed: out of swap space
May  6 08:47:43 freebsd kernel: pid 3787 (postgres), jid 0, uid 770, was killed: out of swap space
May  6 08:47:43 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:49:44 freebsd postgres[4966]: [1-1] 2020-05-06 08:49:44.538 UTC [4966] FATAL:  lock file "postmaster.pid" already exists
May  6 08:49:44 freebsd postgres[4966]: [1-2] 2020-05-06 08:49:44.538 UTC [4966] HINT:  Is another postmaster (PID 3784) running in data directory "/var/db/postgres/data11"?
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd syslogd: last message repeated 2 times
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(31): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd syslogd: last message repeated 1 times
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(24): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(24): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(24): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(24): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(24): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(12): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(24): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(24): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(14): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(11): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(7): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(24): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(14): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(11): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(28): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(25): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(19): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(15): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(12): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(5): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(28): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(25): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(22): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(11): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(25): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(22): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(11): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(25): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(3): failed
May  6 08:52:42 freebsd syslogd: last message repeated 1 times
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(3): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(3): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(3): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(3): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(3): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(3): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(3): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(3): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(20): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(18): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(9): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(19): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(3): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(2): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(7): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd syslogd: last message repeated 1 times
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(12): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(4): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd syslogd: last message repeated 5 times
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(5): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(13): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(32): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(19): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(31): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(21): failed
May  6 08:52:42 freebsd kernel: swap_pager_getswapspace(7): failed
May  6 08:52:45 freebsd kernel: pid 4970 (ruby26), jid 0, uid 0, was killed: out of swap space
May  6 08:52:48 freebsd kernel: swap_pager_getswapspace(32): failed

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info


`su -l git -c "cd /usr/local/www/gitlab-ce && bundle exec rake gitlab:env:info RAILS_ENV=production"`
System information
System:
Current User:   git
Using RVM:      no
Ruby Version:   2.6.5p114
Gem Version:    3.0.6
Bundler Version:2.0.2
Rake Version:   12.3.3
Redis Version:  5.0.8
Git Version:    2.26.2
Sidekiq Version:5.2.8
Go Version:     unknown

GitLab information
Version:        12.9.1
Revision:       Unknown
Directory:      /usr/local/www/gitlab-ce
DB Adapter:     PostgreSQL
DB Version:     11.7
URL:            http://192.168.0.251:10080
HTTP Clone URL: http://192.168.0.251:10080/some-group/some-project.git
SSH Clone URL:  ssh://git@192.168.0.251:2223/some-group/some-project.git
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers:

GitLab Shell
Version:        12.0.0
Repository storage paths:
- default:      /usr/local/git/repositories
GitLab Shell path:              /usr/local/share/gitlab-shell
Git:            /usr/local/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)
(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)
(we will only investigate if the tests are passing)