Skip to content

Make Projects::LicensesController use query cache when retrieving builds the second time around

Summary

Projects::LicensesController performs two similar SQL queries to retrieve the latest builds with License Scanning reports:

  • Pipeline#license_scanning_report (EE) delegates to #latest_report_builds to retrieve builds having a License Scanning report artifact. usage through controller call
  • SCA::LicenseCompliance#latest_build_for_default_branch calls pipeline.builds.latest.license_scan.last to get the latest build having a License Scanning report artifact. usage

Currently, these two SQL queries are slightly different and the query cache isn't triggered. See #327019 (comment 589978602)

Proposal: rewrite either of these functions to align the SQL queries, so that the query cache is used when retrieving License Scanning builds the second time around.

SQL query for Pipeline#latest_report_builds
SELECT
  "ci_builds".*
FROM
  "ci_builds"
WHERE
  "ci_builds"."type" = 'Ci::Build'
  AND "ci_builds"."commit_id" = 84
  AND (
    "ci_builds"."retried" = FALSE
    OR "ci_builds"."retried" IS NULL
  )
  AND (
    EXISTS (
      SELECT
        1
      FROM
        "ci_job_artifacts"
      WHERE
        (ci_builds.id = ci_job_artifacts.job_id)
        AND "ci_job_artifacts"."file_type" IN (10, 101)
    )
  )
/*application:web,correlation_id:01F6SHTBVMQNQCKWD1MTTV1X09,endpoint_id:Projects::LicensesController#index,line:/ee/app/models/ee/ci/pipeline.rb:111:in `block in license_scanning_report'*/

SQL query for SCA::LicenseCompliance#latest_build_for_default_branch
SELECT
  "ci_builds".*
FROM
  "ci_builds"
  INNER JOIN "ci_job_artifacts" ON "ci_job_artifacts"."job_id" = "ci_builds"."id"
WHERE
  "ci_builds"."type" = 'Ci::Build'
  AND "ci_builds"."commit_id" = 84
  AND (
    "ci_builds"."retried" = FALSE
    OR "ci_builds"."retried" IS NULL
  )
  AND "ci_job_artifacts"."file_type" IN (10, 101)
ORDER BY
  "ci_builds"."id" DESC
LIMIT
  1
/*application:web,correlation_id:01F6SHTBVMQNQCKWD1MTTV1X09,endpoint_id:Projects::LicensesController#index,line:/ee/app/models/sca/license_compliance.rb:36:in `block in latest_build_for_default_branch'*/

Improvements

There's a slight performance gain because the SQL query cache is used, and the second SQL query is skipped.

Risks

Involved components

Optional: Intended side effects

Optional: Missing test coverage

Implementation plan

  1. Remove license_scan scope from Build model

  2. This method was used only in SCA::LicenseCompliance, change it to:

    pipeline.latest_report_builds(::Ci::JobArtifact.license_scanning_reports).last
  3. Ensure the tests are passing

  4. Test this fix locally before opening MR to ensure there are no regressions.

Testing

Covered off by feature specs, License Finder tests, Secure Test Projects, E2E tests (master pipeline)

Edited by Adam Cohen