Cache the code quality report to significantly speed up docs pipelines
What does this MR do and why?
With some upcoming plans to change our docs link checker, we've realized that the code quality job is the biggest bottleneck for docs-only pipelines (over 10 minutes while most other jobs are just a few minutes or less). We have been running code quality in every pipeline to make sure MR widgets aren't saying things like "10k new code quality issues" or "10k code quality issues resolved" (paraphrased). This happens when the code quality report is missing in one pipeline, then found in a later one. So we run the job in every pipeline to generate the report, ensuring sure the report is always there.
But today I realized we don't need to generate a new report artifact for code in MRs that don't contain any code. We just need to have a recent report present in the pipeline. The solution: Caching!
This MR:
- Updates the
code_quality:
job which now:- Caches the
gl-code-quality-report.json
report file - Uses a
push
cache policy, so it never tries fetch the cached version, it only saves to the cache. - Sample job: https://gitlab.com/gitlab-org/gitlab/-/jobs/6133248459
- Caches the
- Creates a new
code_quality cache:
job which:- Pulls the most recently cached version of the "code" code quality artifact, which uses the
pull
policy so it never updates the cache. - Uses this cached
gl-code-quality-report.json
for the pipeline's code quality artifact. - Sample job: https://gitlab.com/gitlab-org/gitlab/-/jobs/6133270189
- Pulls the most recently cached version of the "code" code quality artifact, which uses the
- Updates the rules for both jobs such that:
-
code_quality:
Runs in any MR that contains code, but not in docs-only pipelines. If an MR has both code and docs, this job runs. -
code_quality cache:
Does not run in any MR that contains only code, but does run in docs-only pipeline. If an MR has both code and docs, this job does not run.
-
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.