Dependency scanner image exit 1 - without further indication about root cause
Security scanners, among them dependency scanner, rarely show root causes when terminating a job with exit 1. Even if SECURE_LOG_LEVEL is set to 'debug' there is often enough no additional information.
Problem to solve
The problem to solve is to provide as much information as possible. Obviously so many things can go wrong so I need to explain this on an example.
Use case
- A dependency scanner needs to run on a Python application.
- The requirements.txt lists mysqlclient as a package.
- The scanner image for python exits with 1
- Turns out that the scanner image does not provide the underlying required OS package. These need to be installed in a before_script. You hardly can know that this is required nor do you understand that this was the reason to exit 1. Especially because the Python container you're building has a Python base image where these OS level packages belong to the distribution already. As the developer you do not think about what might be missing in the scanner image.
- Turns out the mysqlclient is not a pure Python library. When the package was installed with pip install -r requirements.txt, the METADATA and WHEEL information actually provide the information that this is no pure Python library and also provide information which packages to be installed per apt-get e.g.
Idea
Not all packages might deliver this info per pip install but when they do it would be super helpful to know about this. Mysql is one example, I believe the same is valid for other databases. I was told that not all potentially require OS level packages can be part of the scanner image but maybe more than now. At least the hint to what to look at in detail would be useful.
Imho, the problem exists because pip install is used to find transitive dependencies. If there was an easier way to find these dependencies no install would be needed. There are open source tools which try to do this.
Intended Users
Every user of the dependency scanner
Example project
to be added
-->
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.