.gitlab-ci.yml's Parsed YAML Size Limit - surface information to user so they can manage this
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
Summary
I'm working with a customer in support (GitLab team members can read more in the ticket / SF)
GitLab will refuse to process .gitlab-ci.yml once it exceeds a certain compiled size.
We made the limit configurable for self-managed in Add configurable maximum YAML file size and depth (!68593 - merged) so this is mainly a problem on GitLab.com where this limit can't be changed by folks.
- size: 1MB
- depth: 100
But, our docs state:
- a YAML file that is 200kb is likely to hit the default memory limit.
- The
max_yaml_size_bytesvalue is not directly tied to the size of the YAML file, but rather the memory allocated for the relevant objects.
In other words, from a customer perspective, the limit is arbitrary, and more specifically (quoting the customer)
If we hit this limit (which is guesswork, as I'm not sure how we determine our in memory usage compared to the limit?) we will be totally blocked and unable to run pipelines.
Proposal 1
Could we
- Warn when a CI configuration is, say, 90% of the maximum?
- Display memory use by a CI configuration, in practise?
This second option in particular would improve transparency
The limit is a blunt tool to achieve a broad goal - ensure that a sane amount of compute resource is used to process .gitlab-ci.yml. If folks can see the effect of CI changes on their resource use, they are then able to use that information to write better code.
Perhaps we could do it like code quality, as in: "this Merge request will increase CI memory use by 5% to 754kb. The limit is 1024kb."
Proposal 2
We can investigate migrating from ObjectSpace.memsize_of to .bytesize. bytesize will return the actual size of the YAML file.