## Opportunity The pipeline for building `www-gitlab-com` and deploying to production can take a prohibitively long time for time-sensitive MRs and can discourage collaborators from making small, iterative changes. **NOTE: See also https://gitlab.com/groups/gitlab-org/-/epics/3636** A recent spike in deploy times prompted the creation of a dashboard analyzing [Pipeline metrics on Periscope](https://app.periscopedata.com/app/gitlab/561277/WIP:-www.gitlab.com-Pipeline-Metrics). ## Goals Any improvement to the build time is welcome, but a good goal might be to get the average pipeline duration under 00:10:00. Sub-5:00 times would be optimal. ### Current Performance A rough analysis of some recent master builds highlights specific time spent per job: | Job | Time | | ------ | ------ | | images | 00:06:44 | | blog posts (old) | 00:07:38 | | blog posts new | 00:08:25 | | handbook | 00:09:04 | | js, scss, icons, pdfs | 00:06:51 | | build_proxy_resource_master | 00:07:31 | | deploy | 00:02:09 | | deploy_staging | 00:02:54 | _NOTE: This did not include profiling for the Review App builds, which also add ~ 07:00 on average (but sometimes significantly more) to the pipeline and should be considered for improvement in this Epic_ ## Ideas Issues related to the pipeline performance optimization will be included in this Epic. Some of those might include: * Configuration of a Monorepo with separate projects & pipelines for blog, handbook, jobs, etc * Manually do incremental builds * Use Middleman incremental builds * Make building the Review App optional (opt-out or opt-in?) * More powerful runners for select jobs * Caching cloned repo via docker * Smaller, incremental improvements * Cache node_modules * Short circuit `bundle install` with `bundle check` * https://gitlab.com/gitlab-com/www-gitlab-com/merge_requests/38118 * Eliminate or reduce repeated sub-tasks across jobs * Improving caching policiese * Improve artifact policies * Remove TeX from the Docker image * Implement middleman external pipeline using Webpack or Gulp for assets * Replace `dependencies` with `needs` * Further parallelize build steps This overview is sourced from a detailed [document @cwoolley-gitlab authored](https://docs.google.com/document/d/14oi_d9_tbgLxxkmHUYolRZg725JU7lkkklU-GIO2gDI/edit?usp=sharing)