Pre-create/cache environment for upcoming jobs
Problem to solve
Loading a container in GitLab CI can take a couple minutes. For very large containers (for example, on Windows) this can be very long. As a specific example, the pipeline at https://gitlab.com/jlenny/jobtimer/-/jobs/133203533 which does nothing but sleep for 10 seconds takes 60 seconds in total to execute. https://gitlab.com/jlenny/gitter-android-app/-/jobs/132566449 is an example job where the pipeline took 17 seconds to run the command, but took 2 minutes and 35 seconds in total execution time.
Over the course of a pipeline with a lot of jobs/stages, this can add up to quite a lot of time.
Target audience
Developers/Operations teams that use pipelines and want faster, more efficient execution.
Further details
CircleCI, for comparison, is using caching on job start to accelerate:
Build-agent version 0.1.1216-48f80d08 (2018-12-07T16:01:40+0000)
Starting container circleci/node:8
image is cached as circleci/node:8, but refreshing...Proposal
We can consider pre-creating the environment that a job will run in (pausing when it would fetch artifacts/start execution) in order to accelerate startup time, or look at other options to cache the container image as close to the runner as possible. We would need an analysis of what exactly is happening within the startup time to determine how to accelerate it.
What does success look like, and how can we measure that?
Average time to execute the first command/artifact fetch in a pipeline is reduced by at least half.