Automatically tag and release secure analyzers

Problem to solve

While working on Build docker images on a regular schedule, there was a discussion about whether we should fetch the most recent version of the analyzer from the git tag or from the CHANGELOG.md file. A concern was raised that if a developer forgets to tag and release an analyzer, the most recent version in the CHANGELOG.md file would not match the most recent git tag, since a tag wasn't created. It was decided that we could address this issue, but it was out of scope, so that's why this new issue is being created to track it.

Proposal

The scheduled job for automatically building Docker images (see gemnasium, retire.js, bundler-audit, etc) currently does the following:

1. Fetches the most recent version of the analyzer using git tag -l
2. Rebuilds the analyzer with the version value fetched in step 1.

We should update this process to the following:

1. Fetch the most recent version of the analyzer and the description of the change from the CHANGELOG.md file.

2. Check to see if a tag already exists with the given version value fetched in step 1.:

   • If such a tag exists, then rebuild a new analyzer Docker image with that tag
   • If a tag does not exist, tag a new analyzer Docker image with that tag and provide a tag description which matches the description for the change from the CHANGELOG.md file

Implementation Plan

Create an MR with the following changes, and notify the Static Analysis team about the new CREATE_GIT_TAG variable in case they also want to make use of it:

1. Create a new stage: tag (this contains a job for creating the git tag from the most recent changelog entry if it doesn't exist)

2. Create a new job upsert git tag in the tag stage. Use the code in this commit as a starting point.

   This job is idempotent and will:

   1. Add a new CI variable CREATE_GIT_TAG (create a git tag if missing)
   2. Fetch the most recent version of the analyzer and the description of the change from the CHANGELOG.md file.
   3. If a git tag matching the version from the CHANGELOG.md file does not exist, create a new git tag, provide a tag description which matches the description for the change from the CHANGELOG.md file

   Note: The job is skipped unless CREATE_GIT_TAG is true

3. Update the CI/CD Environment variables following projects to set CREATE_GIT_TAG: true and also configure the GITLAB_TOKEN environment variable to match the one in the ci-templates CI/CD Settings:

   • gemnasium
   • gemnasium-maven
   • gemnasium-python
   • retire.js
   • bundler-audit

Future Steps

1. Increase the frequency of the scheduled CI piplines.

2. Create a new MR with the following changes, and include the Static Analysis team to make sure they approve the changes:

   1. Rename release-major stage to release
   2. Rename major to release major
   3. Rename minor to release minor
   4. Rename tag version to release patch
   5. Merge release-version into release

   See this MR for a starting point for the above changes.

3. Export the major, minor, patch version numbers as CI variables to be used by the release jobs using dotenv export artifact

4. If things go well and Static Analysis agrees with the change, remove the PUBLISH_IMAGES and CREATE_GIT_TAG variables.

   By removing these variables, the behaviour of the default branch build will be to automatically create a new git tag if one doesn't exist, and release the images

Intended users

• Sasha (Software Developer)

Further details

This change will eliminate the need for this issue: Warn when analyzer changelog is updated without a new release

Documentation

• Update the Versioning and release process docs
• Update the Security and Build fixes of Go docs (not necessary - see this discussion for more details)

What does success look like, and how can we measure that?

If a developer forgets to release an analyzer, it will automatically be built and released from a scheduled job. We can test this by updating the changelog entry for an analyzer and ensuring that the scheduled job automatically tags and releases a new version.

What is the type of buyer?

Enterprise Applications GitLab Ultimate

Is this a cross-stage feature?

This change will affect ~"Category:Dependency Scanning". If Category:SAST wants to make use of it, they can do so by setting CREATE_GIT_TAG: "true" and configuring GITLAB_TOKEN to a valid access token.

/cc @NicoleSchwartz @gonzoyumo @fcatteau