Skip to content

Cache the code quality report to significantly speed up docs pipelines

Marcel Amirault requested to merge docs-code-quality-poc-2 into master

What does this MR do and why?

With some upcoming plans to change our docs link checker, we've realized that the code quality job is the biggest bottleneck for docs-only pipelines (over 10 minutes while most other jobs are just a few minutes or less). We have been running code quality in every pipeline to make sure MR widgets aren't saying things like "10k new code quality issues" or "10k code quality issues resolved" (paraphrased). This happens when the code quality report is missing in one pipeline, then found in a later one. So we run the job in every pipeline to generate the report, ensuring sure the report is always there.

But today I realized we don't need to generate a new report artifact for code in MRs that don't contain any code. We just need to have a recent report present in the pipeline. The solution: Caching!

This MR:

  • Updates the code_quality: job which now:
  • Creates a new code_quality cache: job which:
    • Pulls the most recently cached version of the "code" code quality artifact, which uses the pull policy so it never updates the cache.
    • Uses this cached gl-code-quality-report.json for the pipeline's code quality artifact.
    • Sample job: https://gitlab.com/gitlab-org/gitlab/-/jobs/6133270189
  • Updates the rules for both jobs such that:
    • code_quality: Runs in any MR that contains code, but not in docs-only pipelines. If an MR has both code and docs, this job runs.
    • code_quality cache: Does not run in any MR that contains only code, but does run in docs-only pipeline. If an MR has both code and docs, this job does not run.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Edited by Marcel Amirault

Merge request reports