Parallel Downloading of Artifacts in GitLab CI/CD
Summary:
In GitLab CI/CD, when jobs depend on artifacts from multiple previous jobs, the current behavior is to download each artifact sequentially, which can be time-consuming when there are a large number of dependencies or if the artifacts are large. The proposal is to add the ability to download these artifacts in parallel to expedite the setup time for jobs.
This feature would improve the efficiency of GitLab CI/CD pipelines that involve jobs with many or large artifact dependencies.
Desired Outcome: Enhance the artifact download mechanism to support parallel downloads, thereby reducing job setup times in GitLab CI/CD pipelines with multiple or large artifact dependencies.
Related code link: https://gitlab.com/gitlab-org/gitlab-runner/-/blob/main/shells/abstract.go#L223