gemnasium does not always use the latest state/version of gemnasium-db
Release notes
Dependency Scanning is designed to check for the latest version of the vulnerability database (called the gemnasium--db) each time the analyzer is run, however starting [DATE] a bug was introduced and the vulnerability database was only updated when the analyzer was updated (which happens monthly). As a result, since the introduction of the bug, scan results could be identifying only CVE found earlier than one month ago. We are concerned by this bug and will be performing a review of it in order to prevent it, or things similar, from going undetected again.
If you regularly update to the current docker image for Dependency Scanning the bug will be resolved as soon as you are using the latest image. If you are more deliberate in your updates you should X.
Issue
We became aware of the following issue by accident through an advisory request created by @joernchen in this slack discussion.
Summary
The gemnasium dependency scanner leverages the advisory data provided by gemnasium-db which is updated on a daily basis.
There is some logic in the gemnasium dependency scanner for fetching the latest updates from gemnasium-db. As it turned out, gemnasium always fetches the latest updates from gemnasium-db, but the HEAD
is not updated to point to the state of the most recent origin/master
. Therefore, although updates are fetched from gemnasium-db, gemnasium uses the state of gemnasium from the time when the docker image was created.
This affects all Gemnasium-based analyzers: gemnasium
, gemnasium-maven
, and gemnasium-python
.
This regression was introduced when implementing #196868 (closed)
Further details
Gemnasium updates its local git repo of gemnasium-db by running git commands. The Shell equivalent of these comments would be:
git remote set-url origin $GEMNASIUM_DB_REMOTE_URL
git remote update
git checkout $GEMNASIUM_DB_REF_NAME
See repo.go
This code is shared by gemnasium, gemnasium-maven, and gemnasium-python.
Steps to reproduce
You can reproduce the issue by using the sample go project testproject that contains a vulnerability for which an advisory (CVE-2020-29652.yml) was added recently to gemnasium-db (gemnasium-db version: v1.2.383
).
Gemnasium v2.24.0
was released before v1.2.383
based on the older state of gemnasium-db v1.2.377
.
# clone the test project
> git clone https://gitlab.com/julianthome/testadbupdate
> cd testadbupdate
# run gemnasium 2.24.0
> docker run -it --volume "$PWD:/tmp/project" \
--env SECURE_LOG_LEVEL=debug \
--env CI_PROJECT_DIR=/tmp/project \
registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2.24.0
# CVE-2020-29652 not found
> grep -m 1 -o CVE-2020-29652 gl-dependency-scanning-report.json || echo "nothing found"
nothing found
# run the same version of gemnasium but this time by explicitly defining the latest version v1.2.383
> docker run -it --volume "$PWD:/tmp/project" \
--env GEMNASIUM_DB_REF_NAME=v1.2.383 \
--env SECURE_LOG_LEVEL=debug \
--env CI_PROJECT_DIR=/tmp/project \
registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2.24.0
> grep -m 1 -o CVE-2020-29652 gl-dependency-scanning-report.json || echo "nothing found"
CVE-2020-29652
Example Project
The test project that uses the DS template is available under https://gitlab.com/julianthome/testadbupdate. You can see that CVE-2020-29652
does not show up in the vulnerability report. However there are two branches/MR where CVE-2020-29652
shows up in the vulnerability report:
-
https://gitlab.com/julianthome/testadbupdate/-/merge_requests/1: by specifying
GEMNASIUM_DB_REF_NAME: "v1.2.383"
-
https://gitlab.com/julianthome/testadbupdate/-/merge_requests/2: by specifying
GEMNASIUM_DB_REF_NAME: "origin/master"
What is the current bug behavior?
Recently added advisories to gemnasium-db do not show up in the vulnerability report because gemnasium uses the state of gemnasium-db from the time where the docker image was created.
What is the expected correct behavior?
Recently added advisories to gemnasium-db should show up instantaneously in the vulnerability report.
Relevant logs and/or screenshots
You can find more details in the logs for the two jobs that were running on master and the MRs:
[DEBU] [Gemnasium] [2020-12-17T17:27:54Z] ▶ /usr/bin/git -C /gemnasium-db remote set-url origin https://gitlab.com/gitlab-org/security-products/gemnasium-db.git
[DEBU] [Gemnasium] [2020-12-17T17:27:55Z] ▶ /usr/bin/git -C /gemnasium-db remote update
Fetching origin
From https://gitlab.com/gitlab-org/security-products/gemnasium-db
493a7e116..f16104eea master -> origin/master
* [new branch] remove-spellchecker -> origin/remove-spellchecker
* [new tag] v1.2.383 -> v1.2.383
* [new tag] v1.2.380 -> v1.2.380
* [new tag] v1.2.381 -> v1.2.381
* [new tag] v1.2.382 -> v1.2.382
[DEBU] [Gemnasium] [2020-12-17T17:27:55Z] ▶ /usr/bin/git -C /gemnasium-db checkout master
Already on 'master'
Your branch is behind 'origin/master' by 46 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
[DEBU] [Gemnasium] [2020-12-17T17:27:55Z] ▶ /usr/bin/git -C /builds/julianthome/testadbupdate status
HEAD detached at 177fdbe
nothing to commit, working tree clean
- MR1(
✅ ): https://gitlab.com/julianthome/testadbupdate/-/jobs/919387920 - MR2(
✅ ): https://gitlab.com/julianthome/testadbupdate/-/jobs/919403075
[DEBU] [Gemnasium] [2020-12-17T17:37:18Z] ▶ /usr/bin/git -C /gemnasium-db remote set-url origin https://gitlab.com/gitlab-org/security-products/gemnasium-db.git
[DEBU] [Gemnasium] [2020-12-17T17:37:18Z] ▶ /usr/bin/git -C /gemnasium-db remote update
Fetching origin
From https://gitlab.com/gitlab-org/security-products/gemnasium-db
493a7e116..f16104eea master -> origin/master
* [new branch] remove-spellchecker -> origin/remove-spellchecker
* [new tag] v1.2.383 -> v1.2.383
* [new tag] v1.2.380 -> v1.2.380
* [new tag] v1.2.381 -> v1.2.381
* [new tag] v1.2.382 -> v1.2.382
[DEBU] [Gemnasium] [2020-12-17T17:37:18Z] ▶ /usr/bin/git -C /gemnasium-db checkout origin/master
Note: checking out 'origin/master'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at f16104eea Merge branch 'add-CVE-2020-29652' into 'master'
Possible fixes
- use
origin/master
as the default setting for the variableGEMNASIUM_DB_REF_NAME
- revert gitlab-org/security-products/analyzers/gemnasium!73 (merged)
See discussion
Implementation plan
-
Fix problem in gemnasium project gitlab-org/security-products/analyzers/gemnasium!162 (merged) -
Upgrade gemnasium
git submodule in gemnasium-python and release new version -
Upgrade gemnasium
go module in gemnasium-maven and release new version -
Refactor QA jobs in gemnasium; see see gitlab-org/security-products/analyzers/gemnasium!162 (comment 500127325) and gitlab-org/security-products/analyzers/gemnasium!162 (comment 500150971) -
Create an issue to port the solution to bundler-audit, since by designed it's affected by the very same issue; see analyze.go and Dockerfile
Credits
Thanks a lot to @joernchen and @plafoucriere for helping me to understand the nature of this issue.