Skip to content

Allow environment override of NODE_OPTIONS for webpack

DJ Mountney requested to merge nodejs-max-space-override into master

What does this MR do and why?

Allow environment override of NODE_OPTIONS for webpack

  • This should allow 32bit OSes to still compile the assets, even though they sometimes have a 4GB memory access limit

In omnibus we set the limit under 4GB https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/4d913708be53ced2b9c5062f55988ba6c8e138c0/config/software/gitlab-rails.rb#L162 in order for the assets to be able to compile on 32bit PiOS. Even though we run the builds on powerful arm64 32core machines with many GB of RAM, the 32bit PiOS running in a docker image cannot use more than 4GB (at least per process, I haven't verified whether PAE is in place or not).

Note that setting max_old_space_size to 3584 still works for us while compiling assets on a 32bit OS, because nodejs is allocating less memory. Doesn't work for 64bit, with the current size of our assets.

Screenshots or screen recordings

From failing job: https://dev.gitlab.org/gitlab/omnibus-gitlab/-/jobs/17485071

<--- Last few GCs --->
[404806:0x566db40]   865960 ms: Scavenge 848.1 (1037.7) -> 842.9 (1038.4) MB, 40.4 / 1.0 ms  (average mu = 0.007, current mu = 0.006) allocation failure 
[404806:0x566db40]   866861 ms: Scavenge 849.1 (1038.4) -> 845.8 (1038.4) MB, 25.5 / 0.0 ms  (average mu = 0.007, current mu = 0.006) allocation failure 
[404806:0x566db40]   866922 ms: Scavenge 850.3 (1038.4) -> 849.9 (1040.2) MB, 49.0 / 0.0 ms  (average mu = 0.007, current mu = 0.006) allocation failure 
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
Aborted (core dumped)
error Command failed with exit code 134.
Error: Unable to compile webpack production bundle.

How to set up and validate locally

This should fail with heap limit allocation, as our assets can't compile in 500MB

export NODE_OPTIONS="--max_old_space_size=500"
bundle exec rake gitlab:assets:compile

This should pass, our defaults

unset NODE_OPTIONS
bundle exec rake gitlab:assets:compile

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by DJ Mountney

Merge request reports