Pin minor version of SAST, DS analyzers

Problem to solve

When fetching the Docker images of their analyzers, SAST and Dependency Scanning (DS) fetch the tag that matches their MAJOR version. For instance, SAST 2.x fetches the images with tag 2. In a way this is comparable to a automatically upgrading to the latest version of the analyzer that's compatible with SAST or DS. While this automatic upgrade is sometimes convient, this is not what users necessarily want.

Further details

Here's the way it currently works:

  • When fetching the Docker images of their analyzers, SAST and DS use the tag name defined in SAST_ANALYZER_IMAGE_TAG and DS_ANALYZER_IMAGE_TAG, respectively. These environment variables are pre-defined when building the Docker images of SAST and DS.
  • In analyzer projects like Gemnasium, the shared CI configuration builds an image with tag X when the repo is git tagged with X.Y.Z. See https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/pipelines/42829677
  • By convention SAST X.Y is compatible with any analyzer git tagged with X.A.B, and that's why SAST 2.x can fetch Docker images with tag 2.

It's like SAST and DS automatically upgrade all the analyzers they depend on, and they are a few issues with that:

  • the upgrade of the MINOR version (of the analyzer) may result in a behavior change users may not want
  • it's impossible to deploy a new version of an analyzer for a pre-release of GitLab; older versions of GitLab will automatically upgrade too
  • we can't predict the generated reports in old versions of GitLab; they keep changing as we publish new versions of the analyzers

Also, this strategy forces us to release analyzers with a MAJOR number that matches the MAJOR number of the compatible version of SAST or DS, making Semantic Versioning totally impossible.

Proposal

  • Update the CI config shared across analyzers to build images tagged with X.Y.
  • Leverage the vendored job definition of SAST and DS to reference the analyzers with their minor version.

We would have to revisit the environment variables exposed by SAST and DS while maintaining backward compatibility, and clearly this is challenge.

What does success look like, and how can we measure that?

  • developers and maintainers can easily test SAST and DS with specific versions the analyzers
  • users can easily override the pre-defined versions of any analyzer

Links / references

https://gitlab.com/gitlab-org/gitlab-ee/issues/9725 &581 (closed)

/cc @theoretick @plafoucriere @gonzoyumo