MVC Detect vulnerabilities in vendored JavaScript libraries using Gemnasium
Why are we doing this work
Following the design discussions in Design: Detect vulnerabilities in vendored JavaScript libraries using Gemnasium we want to deliver a MVC of scanning capabilities for vendored JS libraries with our Gemnasium analyzer.
Relevant links
Proposal
As a first step, leverage Retire.js to make Gemnasium scan vendored JS libaries. Gemnasium executes retire --js
against the root directory of the git repo being scanned. It does NOT install the project dependencies prior to the scan.
In gemnasium
codebase, We introduce new Go struct type, functions, and methods to:
- Find vendored libraries.
- Find the vulnerabilities matching these vendored libraries, using Gemnasium's vulnerability database.
- Add these vulnerabilities to the Dependency Scanning reports.
We can't reuse the existing high-level functions and type structs because they operate on dependency files, and not vendored libraries. Unlike dep. files, vendored libraries don't have list dependencies, and they're not associated with a package manager.
Please note that vendored libraries are not listed in the dependency_files
field of the Dependency Scanning report b/c they're not dependency files, and shouldn't be presented as such in the UI.
The exact path of a vendored library can be used in the location.file
field of the vulnerabilities, in the report.
Limitations
Gemnasium won't be able to report vulnerabilities for a vendored library if Retire.js doesn't have vulnerabilities for this library, in jsrepository.json
.
Gemnasium accurately report the file path in .vulnerabilities[].location.file
. This was not the case with the retire.js
analyzer though: location.file
was always set to package.json
even when the vulnerable file was part of the git repo. As a result, vulnerability feedback might lost.
Gemnasium doesn't find vulnerable JS files in installed packages. This reduces the noise but it might be seen as a limitation.
Documentation
-
Document the scan of JS libs by Gemnasium. -
Document the CI variable that enables the scan of JS libs in Configuring Dependency Scanning.
Performance
retire --js
takes 40 seconds to process 540 MB and 63k files. See #288330 (comment 813516860)
Testing
Integration tests must cover these scenarios:
- Scan of libraries is enabled.
- Scan of libraries is enabled but some JS files are excluded.
-
retire
command fails. (Error is reported.) - Scan of libraries is NOT enabled.
Implementation plan
To be implemented in the gemnasium project:
-
Add the retire
command to thegemnasium
image via theDockerfile
. -
Introduce a new Go struct type that describes vendored libraries; it combines a file path with a package (type, name, and version). -
Add Go package that finds JS vendored libraries using the retire --js
command. -
Update scanner
package to scan vendored libraries, and report their vulnerabilities. -
Update convert
package to report vulnerabilities found in vendored libraries to the Dependency Scanning report. -
Update main
package to find, scan, and report JS vendored library. -
Add an image test integration to check the scanning of JS vendored libraries. -
Enable the feature in the CI template. -
Update documentation.
🤖
Auto-Summary Discoto Usage
Points
Discussion points are declared by headings, list items, and single lines that start with the text (case-insensitive)
point:
. For example, the following are all valid points:
#### POINT: This is a point
* point: This is a point
+ Point: This is a point
- pOINT: This is a point
point: This is a **point**
Note that any markdown used in the point text will also be propagated into the topic summaries.
Topics
Topics can be stand-alone and contained within an issuable (epic, issue, MR), or can be inline.
Inline topics are defined by creating a new thread (discussion) where the first line of the first comment is a heading that starts with (case-insensitive)
topic:
. For example, the following are all valid topics:
# Topic: Inline discussion topic 1
## TOPIC: **{+A Green, bolded topic+}**
### tOpIc: Another topic
Quick Actions
Action Description /discuss sub-topic TITLE
Create an issue for a sub-topic. Does not work in epics /discuss link ISSUABLE-LINK
Link an issuable as a child of this discussion
Last updated by this job
- TOPIC Documentation #350512 (comment 830861710)
- TOPIC Enabling the scan of vendored libs #350512 (comment 830873069)
- TOPIC Tracking extractors being used #350512 (comment 830888146)
- TOPIC Running npm install #350512 (comment 831007716)
Discoto Settings
---
summary:
max_items: -1
sort_by: created
sort_direction: ascending
See the settings schema for details.