POC: Splitting analyze and build phases in Security Products analyzers
Follow up of #10479 (closed)
Proposal
Implement a Proof of Concept (PoC) that demonstrates extracting the project build from the security scan. This PoC is based on what's been found in #10479 (closed).
-
Select an analyzer project for the PoC -
Update the job definition in order to install the analyzer if not present -
Update the analyzer to skip the build when it's not needed -
Test with a real-world project
See also FAQ
Discussions
The proposal is updated based on the outcome of these discussions:
- bundler-audit PoC
- passing artifacts
- analyzer dependencies
- packaging dependencies
- packaging tools
- "install if not present" on-liner
- CLI name
See also FAQ
Implementation plan
-
Release a Dependency Scanning analyzer project as a Debian package, to be built automatically by the CI pipeline -
Provide a job template that 1. installs the analyzer if not present
and 2.runs the analyzer
- Set up a test project with special system dependencies
-
Case A. Scan this project with a Docker image that's not the analyzer image, and where system dependencies are installed prior to the scan -
Case B. Propagate the build from the build job to the scanning job using CI artifacts
-
To illustrate with Dependency Scanning for Python, the new job definition looks like this:
gemnasium-python-dependency_scanning:
[...]
script:
- [install the analyzer if not present]
- [run the analyzer as usual]
Users can override the base image and set the before_script
so that the scanning job satisfies all the project dependencies (A).
Users can also keep the job definition as is, and pass the output of the build job to the scanning job using job artifacts (B).
What does success look like
Users can easily:
- use their own images for projects when the official analyzer images are incompatible
- install the dependencies in these images, prior to the scan
- use the legacy analyzers and job templates without any change (backward compatibility)
- control the packages and dependencies fetched from the scanning job (air-gapped mode)
Next steps
If the POC is a success, an Epic will be created to organize the adoption of this concept at a larger scale, for all our analyzers. One issue will be dedicated to License Management, which will require a large rewrite. We should go away from the current Docker image, and use something a lot lighter based on this concept (%13.0?). SETUP_CMD
will be deprecated in this epic.