Increase hardcoded defaults for project cloner memory and volume size
MR: Increase memory/disk defaults for workspaces (!162119 - merged)
Description
Customers are reporting workspace failures because the project cloner init container does not have enough memory and/or disk space. See this example Zendesk ticket which shows Reason: OOMKilled
for a cloner injector container crash.
As an immediate solution, we will increase the hardcoded memory and disk limit.
We will also increase limit in ee/lib/remote_development/workspaces/create/tools_component_injector.rb
, to proactively prevent any issues there (256Mi
is kind of low)
Note this does NOT increase the request
values for memory, so the default memory footprint of workspaces should not be increased - this extra memory overhead will only be used if needed.
The larger long-term solution involves giving admins and users more control over these settings, and also over how the cloning is performed. See these related issues:
- Allow configuration of project cloning approach (#414011 - closed), which is an issue to provide more extensive control over the cloning mechanisms.
- Expose settings to control the size of volume w... (&12807), which is about exposing settings for the the memory and volume size of the project cloner init container
- BUG: Errors are not properly handled when proje... (#471531 - closed), which relates to issues on the agent side when cloning fails
Acceptance Criteria
(See Technical Requirements for details)
-
Increase memoryLimit
inproject_cloner_component_injector.rb
andtools_component_injector.rb
-
Increase size
involume_component_injector.rb
-
Update all associated tests and fixture files as necessary.
Technical Requirements
- In
ee/lib/remote_development/workspaces/create/project_cloner_component_injector.rb
andee/lib/remote_development/workspaces/create/tools_component_injector.rb
, increasememoryLimit
from256Mi
to2Gi
. The 'memoryRequest' will remain '128Mi'. - In
ee/lib/remote_development/workspaces/create/volume_component_injector.rb
, increasesize
from15Gi
to50Gi
NOTE: The volume in volume_component_injector.rb
is the same volume used for ALL containers (both the workspaces and the init containers), so we are increasing it 3X to account for any large disk requirements in any of these containers.
Design Requirements
None, no UX facing changes
Impact Assessment
Fixing this will allow the usage of workspaces for customers with larger repos.