<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=15786) </details> <!--IssueSummary end--> GitLab CI is awesome, but not always fast. Let's make it fast, and fast by default. Everyone should be able to have CI run in less than 5 minutes. This is a meta issue to track a variety of ideas for speed improvements. ### Ideas * [ ] Sticky runners (to avoid start time when a runner can be re-used for the same project and to avoid needing to re-download cache and artifacts; re-use common services (e.g. docker daemon)). * [ ] Pre-pull common docker images (to load docker images during boot rather than build start). * [ ] [Build "gitlab/kitchensink" image](gitlab-ce#21626) that contains 90%+ of the tools/binaries that users need (e.g. docker, awscli, python, ruby, kubectl, etc.). * [x] Change default cache key to a string constant so it is shared between branches and jobs: https://gitlab.com/gitlab-org/gitlab-ce/issues/22419 * [ ] Cache known system directories e.g. for ruby gems automatically. * [ ] Start reporting build queue times so people can be aware of delays and look to reduce them. * [ ] Include Timestamps in CI Output: https://gitlab.com/gitlab-org/gitlab-ce/issues/22745 * [ ] Start reporting times for each step in a build (i.e. git fetch, docker pull, etc.) so people can monitor and reduce them. * [ ] Analytics of pipeline runs to help people discover and diagnose speed problems. * [x] Allow users to disable unneeded steps such as git fetch when the job doesn't need the repo. * [x] Formalize parallel jobs: gitlab-ce#21480 * [ ] [Automatically balance parallel jobs](gitlab-ce#3819) (e.g. by file name, by junit data, by knapsack data). * [ ] Improve `.gitlab-ci.yml` templates to use caching and other optimizations. * [ ] Encourage people to use `alpine` * [ ] Provide graphical `.gitlab-ci.yml` editor/creator that helps people create efficient, parallel pipelines: gitlab-ce#21485 * [ ] Enable failing fast (so a pipeline returns failure as soon as any test fails) * [ ] Prioritize tests that tend to fail or have failed recently * [ ] Docker caching. (Image registry doesn't prevent all layers from being rebuild.) * [ ] [Plugins](gitlab-ce#14178) and/or includable templates with good defaults (e.g. `bundle install -n $(nproc)`) * [ ] Set `DOCKER_DRIVER=overlay` by default * [ ] Support massive parallelization (e.g. 500 parallel jobs) efficiently and reliably * [ ] Increase size of docker image proxy cache * [ ] Document information and tips on improving speed of tests and builds * [ ] Cancel a pipeline automatically if the relevant commit no longer exists due to a rebase + force push. * [x] Cancel a pipeline automatically if it's not HEAD: https://gitlab.com/gitlab-org/gitlab-ce/issues/8998 * [x] Cancel a pipeline automatically if a new pipeline for the same branch is started * [ ] Cancel running pipelines automatically if a new pipeline for the same branch is started * [ ] Make it easier to kill an older pipeline after rebasing a merge request * [ ] When a pipeline is queued, show other queued and running builds (and quickly be able to cancel them) * [ ] Pending builds without knowing why * [ ] Inadequate monitoring for all steps of a pipeline * [ ] Pre-seed cache/image/whatever so first build is fast: gitlab-ce#28552 * [ ] Dedicated (paid) persistent Docker daemon (on .com) for faster Docker push/pull/caching * [ ] Inconsistent pipeline performance on GitLab.com * [ ] Run tests in the cloud pre-commit from a client: https://gitlab.com/gitlab-org/gitlab-ce/issues/19267 * [ ] Prioritize previously failed tests: https://gitlab.com/gitlab-org/gitlab-ce/issues/38967 * [ ] Implement something like https://ccache.samba.org/ ### Links / references <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->