[Feature flag] Lower big file threshold
What
Enable the :gitaly_lower_big_file_threshold
feature flag, which will lower the big file threshold from 512MB to 50MB. This has two consequences:
-
Files larger than 50MB are not going to be slurped into memory anymore, but will instead use streaming interfaces.
This should improve memory consumption. -
Files larger than 50MB will not be diffed anymore. This should improve latency for RPCs that compute diffs when any such large files are involved. The downside is that diffs for such files cannot be viewed anymore.
Rolls out Lower `core.bigFileThreshold` so that we stop t... (#4609 - closed), git: Lower big file threshold to improve memory... (!6190 - merged).
Owners
- Team: Gitaly
- Most appropriate slack channel to reach out to:
#g_gitaly
- Best individual to reach out to: pks-gitlab
Expectations
What release does this feature occur in first?
What are we expecting to happen?
We might be able to see a decrease in memory consumption for repositories with large blobs. And second, we might see a decrease in latency for diff-based RPCs for repositories with large blobs.
What might happen if this goes wrong?
We could start to mistreat largish blobs.
What can we monitor to detect problems with this?
- Diff RPCs.
- Memory consumption in RPCs that write packfiles.
Roll Out Steps
-
Enable on staging -
Is the required code deployed on staging? (howto) -
Enable on staging (howto) -
Add featureflagstaging to this issue (howto) -
Test on staging (howto) -
Verify the feature flag was used by checking Prometheus metric gitaly_feature_flag_checks_total
-
-
Enable on production -
Is the required code deployed on production? (howto) -
Progressively enable in production (howto) -
Add featureflagproduction to this issue -
Verify the feature flag was used by checking Prometheus metric gitaly_feature_flag_checks_total
-
-
Default-enable the feature flag (optional, only required if backwards-compatibility concerns exist) -
Wait for release containg default-disabled feature flag. -
Change the feature flag to default-enabled (howto) -
Wait for release containing default-enabled feature flag.
-
-
Remove feature flag
Please refer to the documentation of feature flags for further information.