Dependency Scanning doesn't support NodeJs project using Yarn v2
Summary
Dependency Scanning reports no dependencies for a Node JS project using Yarn v2 is used.
NOTE: This is a multi-project repo that also contains Python code.
Steps to reproduce
- Have a repo that contains Node JS code.
- Dependencies are managed through Yarn v2 for Node
- Run a pipeline that contains the Dependency Scanning job.
- The Node JS dependencies aren't listed in the Dependency List.
Example Project
Repo Structure
├── README.md
├── amplify.yml
├── commitlint.config.js
├── docs
│ └── YSLY.png
├── package.json
├── packages
│ ├── backend
| | └── package.json
│ ├── shared
| | └── package.json
│ ├── types
| | └── package.json
│ └── webapp
| | └── package.json
├── scripts
│ └── exportCognitoUsers
| | └── requirements.txt
└── tsconfig.json
If needed access can be given to the project where the bug is encountered.
What is the current bug behavior?
The Yarn dependencies neither listed nor scanned.
What is the expected correct behavior?
All dependencies are listed and scanned, included the ones managed by Yarn v2.
Relevant logs and/or screenshots
Log of retire-js-dependency_scanning
//
Running with gitlab-runner 13.2.2 (a998cacd)
on docker-auto-scale 0277ea0f
section_start:1596460976:prepare_executor
Preparing the "docker+machine" executor
Using Docker executor with image registry.gitlab.com/gitlab-org/security-products/analyzers/retire.js:2 ...
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image registry.gitlab.com/gitlab-org/security-products/analyzers/retire.js:2 ...
Using docker image sha256:972d44a6f46a7125033b2d641378fd8128cf99d7d07e95b525b26d871e8636e6 for registry.gitlab.com/gitlab-org/security-products/analyzers/retire.js:2 ...
section_end:1596460999:prepare_executor
section_start:1596460999:prepare_script
Preparing environment
Running on runner-0277ea0f-project-18277823-concurrent-0 via runner-0277ea0f-srm-1596460932-d40dbc32...
section_end:1596461003:prepare_script
section_start:1596461003:get_sources
Getting source from Git repository
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/acme-inc/ci-nb/ysly/ysly-monorepo/.git/
Created fresh repository.
Checking out c86b4642 as master...
Skipping Git submodules setup
section_end:1596461005:get_sources
section_start:1596461005:restore_cache
Restoring cache
Checking cache for master...
Downloading cache.zip from https://storage.googleapis.com/gitlab-com-runners-cache/project/18277823/master
Successfully extracted cache
section_end:1596461028:restore_cache
section_start:1596461028:step_script
Executing "step_script" stage of the job script
$ export GITLAB_NPM_REGISTRY_TOKEN=$CI_JOB_TOKEN
$ /analyzer run
[INFO] [retire.js] [2020-08-03T13:23:48Z] â–¶ Detecting project
[INFO] [retire.js] [2020-08-03T13:23:48Z] â–¶ Found project in /builds/acme-inc/ci-nb/ysly/ysly-monorepo
[INFO] [retire.js] [2020-08-03T13:23:48Z] â–¶ Running analyzer
[INFO] [retire.js] [2020-08-03T13:23:48Z] â–¶ node_modules detected, skipping installation.
[INFO] [retire.js] [2020-08-03T13:24:33Z] â–¶ Creating report
section_end:1596461073:step_script
section_start:1596461073:archive_cache
Saving cache
Creating cache master...
node_modules/: found 96193 matching files and directories
Uploading cache.zip to https://storage.googleapis.com/gitlab-com-runners-cache/project/18277823/master
Created cache
section_end:1596461108:archive_cache
section_start:1596461108:upload_artifacts_on_success
Uploading artifacts for successful job
Uploading artifacts...
gl-dependency-scanning-report.json: found 1 matching files and directories
Uploading artifacts as "dependency_scanning" to coordinator... ok id=668407763 responseStatus=201 Created token=FveqLmcW
section_end:1596461110:upload_artifacts_on_success
Job succeeded
Log of gemnasium-dependency_scanning
// gemnasium-dependency_scanning
Running with gitlab-runner 13.2.2 (a998cacd)
on docker-auto-scale 0277ea0f
section_start:1596460976:prepare_executor
Preparing the "docker+machine" executor
Using Docker executor with image registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2 ...
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2 ...
Using docker image sha256:46ee60477c43db056dc8cc4f27ce9c60446d4ff1d976a70f51acf0a93db55d7d for registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2 ...
section_end:1596460999:prepare_executor
section_start:1596460999:prepare_script
Preparing environment
Running on runner-0277ea0f-project-18277823-concurrent-0 via runner-0277ea0f-srm-1596460933-9253f0d9...
section_end:1596461003:prepare_script
section_start:1596461003:get_sources
Getting source from Git repository
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/acme-inc/ci-nb/ysly/ysly-monorepo/.git/
Created fresh repository.
Checking out c86b4642 as master...
Skipping Git submodules setup
section_end:1596461006:get_sources
section_start:1596461006:restore_cache
Restoring cache
Checking cache for master...
Downloading cache.zip from https://storage.googleapis.com/gitlab-com-runners-cache/project/18277823/master
Successfully extracted cache
section_end:1596461029:restore_cache
section_start:1596461029:step_script
Executing "step_script" stage of the job script
$ export GITLAB_NPM_REGISTRY_TOKEN=$CI_JOB_TOKEN
$ /analyzer run
[INFO] [gemnasium] [2020-08-03T13:23:49Z] â–¶ Found project in /builds/acme-inc/ci-nb/ysly/ysly-monorepo
section_end:1596461030:step_script
section_start:1596461030:archive_cache
Saving cache
Creating cache master...
node_modules/: found 96193 matching files and directories
Uploading cache.zip to https://storage.googleapis.com/gitlab-com-runners-cache/project/18277823/master
Created cache
section_end:1596461068:archive_cache
section_start:1596461068:upload_artifacts_on_success
Uploading artifacts for successful job
Uploading artifacts...
gl-dependency-scanning-report.json: found 1 matching files and directories
Uploading artifacts as "dependency_scanning" to coordinator... ok id=668407761 responseStatus=201 Created token=nsHK2zzH
section_end:1596461069:upload_artifacts_on_success
Job succeeded
The dependency list only contains Python dependencies, except for 2 vendored Node.js libraries found in node_modules
.
Dependencies returned by the API (JSON)
{
"report": {
"status": "ok",
"job_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/builds/668407763",
"generated_at": "2020-08-03T13:25:10.432Z"
},
"dependencies": [
{
"name": "appdirs",
"packager": "Python (pip)",
"version": "1.4.3",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "attrs",
"packager": "Python (pip)",
"version": "19.3.0",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "boto3",
"packager": "Python (pip)",
"version": "1.12.42",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "botocore",
"packager": "Python (pip)",
"version": "1.15.42",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "certifi",
"packager": "Python (pip)",
"version": "2019.11.28",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "colorama",
"packager": "Python (pip)",
"version": "0.4.3",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "distlib",
"packager": "Python (pip)",
"version": "0.3.0",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "docutils",
"packager": "Python (pip)",
"version": "0.15.2",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "filelock",
"packager": "Python (pip)",
"version": "3.0.12",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "ijson",
"packager": "Python (pip)",
"version": "2.5.1",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "importlib-metadata",
"packager": "Python (pip)",
"version": "1.5.0",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "importlib-resources",
"packager": "Python (pip)",
"version": "1.4.0",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "isort",
"packager": "Python (pip)",
"version": "4.3.21",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "jmespath",
"packager": "Python (pip)",
"version": "0.10.0",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "jquery",
"packager": "",
"version": "2.0.0",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/node_modules/faker/locale/.publish/scripts/prettify/jquery.min.js",
"path": "node_modules/faker/locale/.publish/scripts/prettify/jquery.min.js"
},
"vulnerabilities": [
{
"name": "parseHTML() executes scripts in event handlers in jquery",
"severity": "medium"
},
{
"name": "3rd party CORS request may execute in jquery",
"severity": "medium"
},
{
"name": "Regex in its jQuery.htmlPrefilter sometimes may introduce XSS in jquery",
"severity": "medium"
},
{
"name": "jQuery before 3.4.0, as used in Drupal, Backdrop CMS, and other products, mishandles jQuery.extend(true, {}, ...) because of Object.prototype pollution in jquery",
"severity": "low"
}
],
"licenses": []
},
{
"name": "jquery",
"packager": "",
"version": "1.9.1",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/node_modules/tinycolor2/demo/jquery-1.9.1.js",
"path": "node_modules/tinycolor2/demo/jquery-1.9.1.js"
},
"vulnerabilities": [
{
"name": "parseHTML() executes scripts in event handlers in jquery",
"severity": "medium"
},
{
"name": "3rd party CORS request may execute in jquery",
"severity": "medium"
},
{
"name": "Regex in its jQuery.htmlPrefilter sometimes may introduce XSS in jquery",
"severity": "medium"
},
{
"name": "jQuery before 3.4.0, as used in Drupal, Backdrop CMS, and other products, mishandles jQuery.extend(true, {}, ...) because of Object.prototype pollution in jquery",
"severity": "low"
}
],
"licenses": []
},
{
"name": "jsonschema",
"packager": "Python (pip)",
"version": "3.1.1",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "packaging",
"packager": "Python (pip)",
"version": "19.2",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "pip",
"packager": "Python (pip)",
"version": "20.1.1",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [
{
"name": "Improper Input Validation in pip",
"severity": "high"
}
],
"licenses": []
},
{
"name": "pipdeptree",
"packager": "Python (pip)",
"version": "0.13.2",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "pipenv",
"packager": "Python (pip)",
"version": "2018.11.26",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "pyparsing",
"packager": "Python (pip)",
"version": "2.4.6",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "pyrsistent",
"packager": "Python (pip)",
"version": "0.15.7",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "python-dateutil",
"packager": "Python (pip)",
"version": "2.8.1",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "s3transfer",
"packager": "Python (pip)",
"version": "0.3.3",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "setuptools",
"packager": "Python (pip)",
"version": "47.3.1",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "six",
"packager": "Python (pip)",
"version": "1.14.0",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "urllib3",
"packager": "Python (pip)",
"version": "1.25.10",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "virtualenv",
"packager": "Python (pip)",
"version": "20.0.13",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "virtualenv-clone",
"packager": "Python (pip)",
"version": "0.5.3",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "wheel",
"packager": "Python (pip)",
"version": "0.34.2",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
},
{
"name": "zipp",
"packager": "Python (pip)",
"version": "3.1.0",
"location": {
"blob_path": "/acme-inc/ci-nb/ysly/ysly-monorepo/-/blob/c86b46426ac63c13958de9ad120a6ec8a0a45c27/scripts/exportCognitoUsers/requirements.txt",
"path": "scripts/exportCognitoUsers/requirements.txt"
},
"vulnerabilities": [],
"licenses": []
}
]
}
The project uses Yarn v2 AKA yarn berry
// .yarnrc.yml
...
nodeLinker: node-modules
plugins:
- path: .yarn/plugins/@yarnpkg/plugin-workspace-tools.cjs
spec: "@yarnpkg/plugin-workspace-tools"
- path: .yarn/plugins/@yarnpkg/plugin-interactive-tools.cjs
spec: "@yarnpkg/plugin-interactive-tools"
yarnPath: .yarn/releases/yarn-berry.js
Output of checks
This bug happens on GitLab.com, I don't know if this happens on self-managed as well.
Possible fixes
(If you can, link to the line of code that might be responsible for the problem)
Implementation plan
-
[ ] investigate support for yarn v2 projects, add a corresponding FREEZE branch to our relevant test projects to validate this.moved to #263358 (closed) -
update of the documentation to clarify we do not support yarn v2 yet -
investigate the situation of this specific project (request access to example project)