Automatically tag and release secure analyzers
Problem to solve
While working on Build docker images on a regular schedule, there was a discussion about whether we should fetch the most recent version of the analyzer from the git tag
or from the CHANGELOG.md
file. A concern was raised that if a developer forgets to tag and release an analyzer, the most recent version in the CHANGELOG.md
file would not match the most recent git tag
, since a tag wasn't created. It was decided that we could address this issue, but it was out of scope, so that's why this new issue is being created to track it.
Proposal
The scheduled job for automatically building Docker images (see gemnasium, retire.js, bundler-audit, etc) currently does the following:
- Fetches the most recent version of the analyzer using
git tag -l
- Rebuilds the analyzer with the version value fetched in step
1.
We should update this process to the following:
-
Fetch the most recent version of the analyzer and the description of the change from the
CHANGELOG.md
file. -
Check to see if a tag already exists with the given version value fetched in step
1.
:- If such a tag exists, then rebuild a new analyzer Docker image with that tag
- If a tag does not exist, tag a new analyzer Docker image with that tag and provide a tag description which matches the description for the change from the
CHANGELOG.md
file
Implementation Plan
Create an MR with the following changes, and notify the Static Analysis team about the new CREATE_GIT_TAG
variable in case they also want to make use of it:
-
Create a new stage:
tag
(this contains a job for creating the git tag from the most recent changelog entry if it doesn't exist) -
Create a new job
upsert git tag
in thetag
stage. Use the code in this commit as a starting point.This job is idempotent and will:
- Add a new CI variable
CREATE_GIT_TAG
(create a git tag if missing) - Fetch the most recent version of the analyzer and the description of the change from the
CHANGELOG.md
file. - If a git tag matching the version from the
CHANGELOG.md
file does not exist, create a new git tag, provide a tag description which matches the description for the change from theCHANGELOG.md
file
Note: The job is skipped unless
CREATE_GIT_TAG
is true - Add a new CI variable
-
Update the
CI/CD Environment variables
following projects to setCREATE_GIT_TAG: true
and also configure theGITLAB_TOKEN
environment variable to match the one in the ci-templatesCI/CD Settings
:
Future Steps
-
Increase the frequency of the scheduled CI piplines.
-
Create a new MR with the following changes, and include the Static Analysis team to make sure they approve the changes:
- Rename
release-major
stage torelease
- Rename
major
torelease major
- Rename
minor
torelease minor
- Rename
tag version
torelease patch
- Merge
release-version
intorelease
See this MR for a starting point for the above changes.
- Rename
-
Export the
major
,minor
,patch
version numbers as CI variables to be used by therelease
jobs using dotenv export artifact -
If things go well and Static Analysis agrees with the change, remove the
PUBLISH_IMAGES
andCREATE_GIT_TAG
variables.By removing these variables, the behaviour of the default branch build will be to automatically create a new git tag if one doesn't exist, and release the images
Intended users
Further details
This change will eliminate the need for this issue: Warn when analyzer changelog is updated without a new release
Documentation
- Update the Versioning and release process docs
-
Update the Security and Build fixes of Go docs(not necessary - see this discussion for more details)
What does success look like, and how can we measure that?
If a developer forgets to release an analyzer, it will automatically be built and released from a scheduled job. We can test this by updating the changelog entry for an analyzer and ensuring that the scheduled job automatically tags and releases a new version.
What is the type of buyer?
Enterprise Applications GitLab Ultimate
Is this a cross-stage feature?
This change will affect ~"Category:Dependency Scanning". If Category:SAST wants to make use of it, they can do so by setting CREATE_GIT_TAG: "true"
and configuring GITLAB_TOKEN
to a valid access token.