Skip to content

Insufficient memory limits for webpack cause OOM on from-source installs/upgrades

Summary

Attempting to upgrade a from source installation will fail due to insufficient free memory available to webpack.

Fix is to align instructions with lib/tasks/gitlab/assets.rake which uses 8192 vs. 4096 (not sure why the Rake task only applies on CI systems, removing that limitation and removing the --max_old_space_size from the commands in the instructions would remove the need to update this each time the memory limit is hit)

Steps to reproduce

  1. Have an outdated GitLab installation and follow steps to upgrade it
  2. Run the step for rebuilding assets and observe it fails

Example Project

Not relevant, issue with core GitLab product.

What is the current bug behavior?

# bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production NODE_OPTIONS="--max_old_space_size=4096"

(lines omitted)

<--- Last few GCs --->

[3811251:0x1aa40d40]   619881 ms: Scavenge 4046.8 (4140.1) -> 4046.4 (4142.1) MB, 6.01 / 0.00 ms  (average mu = 0.991, current mu = 0.989) allocation failure; 
[3811251:0x1aa40d40]   621334 ms: Scavenge 4047.5 (4142.1) -> 4047.0 (4140.6) MB, 5.50 / 0.00 ms  (average mu = 0.991, current mu = 0.989) allocation failure; 
[3811251:0x1aa40d40]   621340 ms: Scavenge 4047.8 (4140.6) -> 4047.4 (4145.6) MB, 4.79 / 0.00 ms  (average mu = 0.991, current mu = 0.989) allocation failure; 


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0xb76dc5 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [webpack]
 2: 0xee6020 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [webpack]
 3: 0xee6307 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [webpack]
 4: 0x10f7f55  [webpack]
 5: 0x110fdd8 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [webpack]
 6: 0x10e5ef1 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [webpack]
 7: 0x10e7085 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [webpack]
 8: 0x10c46d6 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [webpack]
 9: 0x1520216 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [webpack]
10: 0x7fe1df699ef6 
Aborted (core dumped)
error Command failed with exit code 134.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Error: Unable to compile webpack production bundle.

What is the expected correct behavior?

Build should succeed.

Relevant logs and/or screenshots

Posted above

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info

System information
System:         Ubuntu 22.04
Current User:   git
Using RVM:      no
Ruby Version:   3.2.9
Gem Version:    3.7.1
Bundler Version:2.7.1
Rake Version:   13.0.6
Redis Version:  8.2.2
Sidekiq Version:7.3.9
Go Version:     go1.25.1 linux/amd64

GitLab information
Version:        18.4.1
Revision:       a908418d739
Directory:      /home/git/gitlab
DB Adapter:     PostgreSQL
DB Version:     16.10
URL:            https://invent.kde.org
HTTP Clone URL: https://invent.kde.org/some-group/some-project.git
SSH Clone URL:  git@invent.kde.org:some-group/some-project.git
Using LDAP:     yes
Using Omniauth: no

GitLab Shell
Version:        14.45.2
Repository storages:
- default:      unix:/home/git/gitlab/tmp/sockets/private/gitaly.socket
GitLab Shell path:              /home/git/gitlab-shell

Gitaly
- default Address:      unix:/home/git/gitlab/tmp/sockets/private/gitaly.socket
- default Version:      14.3.0-rc1-12422-g7584fbc5054
- default Git Version:  2.50.1

git@lerwini:~/gitlab$ nodejs --version
v20.19.5
git@lerwini:~/gitlab$ yarn --version
1.22.22

Results of GitLab application Check

Not relevant for this issue

Possible fixes

Updates to documentation needed to:

  • doc/update/patch_versions.md
  • doc/install/self_compiled/_index.md
  • doc/update/upgrading_from_source.md
  • doc/update/convert_to_ee/self_compiled.md

I would have submitted a MR for this myself, however i'm unable to do so as forking gitlab-org/gitlab results in a >10GB repository apparently (looks like no de-duplication??) which puts it over the free limits and makes the repository immediately read-only (preventing me from committing to my fork and therefore making any commits to prepare a MR)

Edited by 🤖 GitLab Bot 🤖