Add Python 3 to the retire.js analyzer, make Python 2 optional
Problem to solve
Follow-up to https://gitlab.com/gitlab-org/gitlab-ee/issues/11413
When running npm install
, the retire.js
analyzer plugin is unable to install native modules/addons(packages that build with node-gyp
).
We should also ensure that the other necessary prerequisites are installed for node-gyp
: https://github.com/nodejs/node-gyp#on-unix
Intended users
Further details
This happens because python
is not the part of node:11-alpine
base image upon which the retire.js
analyzer plugin image is built.
Because of this, we're unable to run npm install
for projects with native modules/addons dependencies like Gitter.
https://gitlab.com/gitlab-org/gitter/webapp/-/jobs/235354697
Installing dependencies...
> heapdump@0.3.12 install /tmp/app/node_modules/heapdump
> node-gyp rebuild
gyp ERR! configure error
gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
gyp ERR! stack at PythonFinder.failNoPython (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:484:19)
gyp ERR! stack at PythonFinder.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:406:16)
gyp ERR! stack at F (/usr/local/lib/node_modules/npm/node_modules/which/which.js:68:16)
gyp ERR! stack at E (/usr/local/lib/node_modules/npm/node_modules/which/which.js:80:29)
gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/which/which.js:89:16
gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/isexe/index.js:42:5
gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/isexe/mode.js:8:5
gyp ERR! stack at FSReqCallback.oncomplete (fs.js:158:21)
Proposal
-
leverage these variables in the gemnasium-python analyzer -
Add python
to theDockerfile
of theretire.js
analyzer plugin image. -
Allow users to set Python 2 or 3 as part of this solution. We can implement a DS_PYTHON_VERSION
variable similar toLM_PYTHON_VERSION
done in (https://gitlab.com/gitlab-org/security-products/license-management/merge_requests/36). Default version should be3
. -
Add DS_PYTHON_VERSION
to our vendored template to pass them down to the analyzers.
Permissions and Security
No special permissions
Documentation
-
Add the new variable in Dependency Scanning documentation: https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html#available-variables. We probably need to specify that only the retire.js
analyzer is supporting this option (and not thegemnasium-python
one).
Testing
-
Native modules/addons dependency should be added to js-npm
test project -
Native modules/addons dependency should be added to js-yarn
test project
What does success look like, and how can we measure that?
The number of customers that have problems with npm
-based projects with native modules/addons dependencies.
What is the type of buyer?
- Internal customers
- GitLab Ultimate users
Links / references
cc @vzagorodny