Tune parallel merge trains concurrency
Status
- Changed concurrency level from 4 to 20 for www-gitlab-com project at 2019-10-30 01:24 UTC
Problem to solve
We introduced parallel merge trains via https://gitlab.com/gitlab-org/gitlab-ee/issues/11222. We set a limitation of four concurrent running pipelines, as documented at https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/#requirements-and-limitations, in order to ensure we didn't run too many concurrent pipelines optimistically, resulting in wasted runner minutes
It's possible we're actually optimizing for the wrong thing here, and probable that we're at least being too conservative. Developer time is much more expensive than runner minutes, and we should prioritize fast results in busy projects over the small incremental cost of runner minutes.
Intended users
Users of merge trains
Further details
Proposal
We should remove the limitation on concurrency, or at least set it to a "protect the system" value rather than something as small as 4 which is realistic under real-world conditions in busy projects.
Some factors that should be considered that might determine the final value:
- How long have we seen the queue grow in production? That might give us an idea what a realistic limitation is that protects the platform but normal users won't bump into.
- How often do merges fail in a merge train? If this is low, and it probably is, it makes a case for as high as possible setting.
Permissions and Security
TODO
-
Replace !19131 (merged) by a proper fix.