Vulnerabilities reported by Gemnasium don't have a severity. Not only this information is important to users, but it's also very much needed to sort the vulnerabilities in the Security Dashboard.
The YAML files of gemnasium-db should contain a severity field or a field from which the severity can be calculated, to be propagated by gemnasium to the GitLab backend via the Dependency Scanning report.
Proposal
Add CVSS vectors (v2 and v3) to the YAML schema of gemnasium-db and make gemnasium convert these to a severity when generating the report.
Both CVSS vectors are optional, and CVSS v3 is preferred.
The severity of a vulnerability reported by Gemnasium is:
the textual representation of the CVSS v3 base score, if defined
else the textual representation of the CVSS v2 base score, if defined
This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
We should probably discuss it with the team as I haven't played with Gemnasium for a while now and I'm not sure about all the layers that would be impacted but it certainly will be from DB to API so it involves quite some work. Also, we should also consider to get that data for advisories already present in the DB, which may be a painful task... We could also normalize identifiers so that we could easily automate such update with a script (for some identifiers like CVEs it would help).
@leipert hard to tell before reviewing the sources that provide us this data. As we already cover a lot of possible values I'd say it's unlikely to be the case but who knows...
Also, we should also consider to get that data for advisories already present in the DB, which may be a painful task... We could also normalize identifiers so that we could easily automate such update with a script (for some identifiers like CVEs it would help).
@leipert There's nothing to be done in GitLab itself: the Gemnasium Analyzer for SAST will set the severity of the vulnerabilities in the reports (when available), and that's something the frontend already handles in a generic way.
We could either store the CVSS v2 (float), the CVSS v3 (float), or a severity ranging from low to critical (enum), or a combination of these. rubysec accepts both CVSS v2 and CVSS v3 in its YAML files, which seems to be a good approach unless we don't know enough about the vulnerability to be that accurate.
@gonzoyumo@plafoucriere I recommend we accept all three (CVSS v2, CVSS v3, severity) in the YAML files of gemnasium-db but convert it to a generic severity column (enum) to store it in the database. This is practical: it responds to our immediate needs (i.e. generate reports with a severity fields), it allows for poorly described advisories, and it makes possible to upgrade to CVSS scores in the future if relevant. cc @dappelt
@dappelt Yes, we could probably accept the full CVSS vector, at least in the YAML. Then we can turn this vector into a single CVSS score to be stored in the DB, or even a level ranging from low to critical. Again, storing the most accurate data in the YAML files makes it possible to improve the DB later on.
Since CVSS v3 was introduced in 2012, old advisories like CVE-2011-5036 only have a CVSS v2 vector.
These days they have both v2 & v3 vectors,
see CVE-2019-12308.
Since we may store the full CVSS vector I've considered converting all CVSS v2 vectors to v3 vectors.
But reading about the changes between the two versions I came to the conclusion that there was no satisfactory way of doing this automatically. Same conclusion in this thread.
At this point I suggest we store the full CVSS vectors in the DB using composite types. Both CVSS vectors v2 and v3 would be supported. There would be no immediate benefit because the existing report syntax is limited and only accepts a severity that ranges from low to critical. But in the future we could leverage these vectors:
to improve the UI and present the vectors like on NVD
to make it possible to query specific values of these vectors
@gonzoyumo It's worth saying that storing the CVSS vectors in the DB is more complex and thus more expensive, regarding the time spent implementing the future. We could iterate and first store a severity column, and then store the full CVSS vectors. They are cons though:
If stored in the DB, the CVSS vectors would be converted to base scores, and ultimately to a normalized severity ranging from low to critical. Since there's only one severity in the report, the CVSS v3 base score would win over CVSS v2. The two conversions would either happen in the DB or in the Gemnasium Analyzer generating the report - to be defined later.
It remains to be decided whether the DB schema makes it possible to store a severity in the case where the CVSS vectors are unknown. I've changed my mind and now think we don't need that since CVEs published on NVD always have CVSS vectors. But that is debatable. @dappelt@plafoucriere WDYT?
I recommend we accept all three (CVSS v2, CVSS v3, severity) in the YAML files of gemnasium-db but convert it to a generic severity column (enum) to store it in the database.
@plafoucriere has raised concerns about possible inconsistencies between the DB schema and the YAML schema, saying that it would be confusing. Fair enough!
@gonzoyumo About starting with severity versus CVSS vectors, I'm not pushing in any direction. It seems reasonable to iterate and repeat the same work - kind of - whenever we're ready to add CVSS vectors to the Gemnasium DB. At least we'll have step-by-step instructions on how to proceed to add a new field to the advisories, using the first batch of MRs as examples. WDYT?
I'd say that as we currently don't use CVSS anywhere, we'd better ask for just a severity to be mandatory for now. But can accept optional attributes for CVSS scores as it's costless to support them in the yaml syntax. Syncing gemnasium-db with our PG database later to leverage these CVSS scores will be easy.
The only issue to me is to make sure we agree on conversion from CVSS to severity but it looks like this is well defined already as you already pointed. Then make sure this maps correctly with our own severity values in rails backend (but this is the job of the analyzers to me, like any other tool). We also could have to update the values we accept in our DB to match other cases than CVSS score (if there are some), but this looks out of scope to me.
@plafoucriere has raised concerns about possible inconsistencies between the DB schema and the YAML schema, saying that it would be confusing. Fair enough!
Could you elaborate on this? I don't see the problem to have the yaml being richer than our DB
At this point I suggest we store the full CVSS vectors in the DB using composite types. Both CVSS vectors v2 and v3 would be supported. There would be no immediate benefit because the existing report syntax is limited and only accepts a severity that ranges from low to critical.
And as per the follow-up I like making v2 and v3 optional (as older v2 don't have v3 and maybe newer v3 after a point lack v2 data) but severity is required and matches our set of options
right now we are using severity on this front end (as best i can tell) most places and i like the idea of consistency, and we can include extra data in details areas if UX wants and if we have it
Several metrics were changed, added, and removed. The numerical formulas were updated to incorporate the new metrics while retaining the existing scoring range of 0-10. Textual severity ratings of None (0), Low (0.1-3.9), Medium (4.0-6.9), High (7.0-8.9), and Critical (9.0-10.0)[9] were defined, similar to the categories NVD defined for CVSS v2 that were not part of that standard .[10]
Same conversion applies to CVSS v2 base score.
This textual severity is similar to the severity defined in the report JSON format of GitLab Dependency Scanning:
How much the vulnerability impacts the software. Possible values: Undefined (an analyzer has not provided this info), Info, Unknown, Low, Medium, High, Critical.
The only exception is "none" but it could be converted to "info". That said, to me it makes no sense to publish a security advisory if the severity is "none". Anyways, it maps well.
I'd say that as we currently don't use CVSS anywhere, we'd better ask for just a severity to be mandatory for now. But can accept optional attributes for CVSS scores as it's costless to support them in the yaml syntax. Syncing gemnasium-db with our PG database later to leverage these CVSS scores will be easy.
@gonzoyumo Then I would make "[accepting] attributes for CVSS scores" a follow up issue. We may decide to introduce CVSS vectors instead, so at least this follow up issue will give a context to discuss this. Also, the follow up issue makes possible for @plafoucriere to challenge what would be a mismatch between the DB schema and the YAML schema. Do you agree on focusing on severity and creating this follow up issue? cc @NicoleSchwartz
@plafoucriere has raised concerns about possible inconsistencies between the DB schema and the YAML schema, saying that it would be confusing. Fair enough!
I don't like having unnecessary ETL rules, that we will have to keep in mind in the future. The DB should carry raw data, and it's only when we query it that we should transform it if needed (like for example we store a CVSS vector, and the output is a severity string from low to high).
The only exception is "none" but it could be converted to "info". That said, to me it makes no sense to publish a security advisory if the severity is "none".
Treating vulnerabilities with severity "none" as an "informative" finding sounds good to me. In fact, this is what the Security team is doing when we get HackerOne reports that have minimal impact.
I've opened 6 MRs in WIP state for this issue, out of 8. The two remaining MRs are to update gemnasium-maven and gemnasium-python, but that doesn't make sense at this point since the analyzers/gemnasium project needs to be updated first.
More MRs have to be created in the test projects once the code is ready.
This issue does NOT cover catching up and setting the severity of the advisories that already are in Gemnasium DB.
@gonzoyumo I'd say catching up is not so expensive, especially if we simply store the severity, and we should to it. The update would be twofold:
update the YAML files gemnasium-db repo (one merge request)
update the Gemnasium DB (one SQL query)
All the data needed to set the severity (or the CVSS vectors) is available on NVD. The JSON exports could be leveraged to extract the information for all the CVE ids found in the Gemnasium DB.
Great news. We still need to figure it out for vulns with other identifiers than CVE but that could be a great first step. Please keep that for the end though, as even if doable, this is less important than shipping the whole feature and other priorities for 12.1.
@gonzoyumo@NicoleSchwartz I'd say that adding severities to the existing advisories is more important than accepting CVSS scores or CVSS vectors we won't be able to present in the short term. Do we agree on this?
Any follow ups we slice off should get their own issues for 12.2/12.3
Another customer today wants to apply an environmental factor to CVSS to determine their own severity level using official CVSS calculator.
cc: @simonwill
With #14630 (closed), most of the tasks in the description should be obsolete, and it would be actually very simple to achieve this issue afterward. We should consider its prioritization again.
@julianthome Is there a way to catch-up on gemnasium-db for this? Maybe we can script something to extract CVSS data from NVD or another feed? We often have a good source to explore in the list of related links for each advisory.
@julianthome Is there a way to catch-up on gemnasium-db for this? Maybe we can script something to extract CVSS data from NVD or another feed?
For all the advisories that can be linked to CVEs (about half of all the advisories contained in gemnasium-db), we can take this information directly from the NVD feed. For the other half, I have to check whether there is a way to at least semi-automate the process.
Ideally the YAML files would accept both CVSS vectors and severities.
Alternatively, we could also say that we only support CVSS vectors and that contributors, if the CVSS for a certain vulnerability is not provided, could use the CVSS calculator to compute it. This would reduce the level of subjectiveness when determining the severity of a vulnerability to a certain extent. However, this also implies some manual work on the advisories in gemnasium-db that are originating from sources other than NVD because we do not have the CVSS vectors for them.
Maybe I am leaning towards proposal B because it makes the adoption of CVSS as our standard vulnerability scoring system more explicit to human contributors and provides, as @fcatteau explained in this comment, the raw, non-degraded signal. It also has the advantage of making us more flexible in case we change GitLab's vulnerability severity scheme.
@julianthome@NicoleSchwartz Thank you both for your feedback! Let's move forward, and make Proposal B the official proposal for this issue. I'm updating the issue. I'm also updating my initial comment with the two proposals, so that we don't loose them. cc @gonzoyumo
@gonzoyumo To me we should move this out of the issue, so that I can estimate the weight related to the changes in the Gemnasium analyzer. Or should we create a sub-issue? Please advise.
I agree and would not use sub-issue here since it's not about a task to be achieved by a counterpart of the same group.
To me, In adbcurate, collect the CVSS vectors when importing advisories from NVD is clearly a separate task to be achieved by a different team and that is blocking this one.
That being said, separation of ~"Category:Vulnerability Database" into a different group is still pending so
To me, In adbcurate, collect the CVSS vectors when importing advisories from NVD is clearly a separate task to be achieved by a different team and that is blocking this one.
Since the data fields for the CVSS vectors are added in this MR, we have a sort of agreed interface between gemnasium-db and gemnasium. Therefore the progress of the item:
In gemnasium, update the advisory.Advisory Go struct to load the CVSS vectors, and convert CVSS v3 and v2 vectors to severity when generating the report
is not really blocked anymore by gitlab-org/security-products/advisory-db-curation-tools#16 I suppose.
However, I agree that the severity feature would not be really useful without the actual data . I am in the process of adapting adbcurate to provide CVSS data so that, as of tomorrow, newly created advisories will provide CVSS vectors.
@gonzoyumo I've updated the issue with a detailed implementation plan. To me weight is 3, and this is a conservative value. It's ready for development.
@NicoleSchwartz Like @gonzoyumo said, this is workflowready for development, it is affordable, and it has direct user value. Adding the severity to the vulnerability findings reported by Gemnasium would have a significant impact on the Security Dashboard. So is this coming soon? %12.9 or %12.10 maybe?
I am considering for 12.10 but I may delay due to the changes in the word "severity" and "confidence"
@NicoleSchwartz thanks. There is no blocker, we'll just make sure the Gemnasium analyzer won't use the Undefined value for Severity. The issue's description already states to use the Unknown value.
Nicole Schwartzchanged title from Add Severity level to Gemnasium vulnerabilities to Add Severity level to vulnerabilities found by Gemnasium
changed title from Add Severity level to Gemnasium vulnerabilities to Add Severity level to vulnerabilities found by Gemnasium
Nicole Schwartzchanged title from Add Severity level to vulnerabilities found by Gemnasium to Provide Severity levels for vulnerabilities found by Gemnasium
changed title from Add Severity level to vulnerabilities found by Gemnasium to Provide Severity levels for vulnerabilities found by Gemnasium
Nicole Schwartzchanged the descriptionCompare with previous version