It is difficult for users and GitLab staff to understand what is and is not currently inside the Vulnerability Database and information about how often this information is updated.
It is difficult for users to understand when a new vulnerability is announced, if GitLab is already scanning for it or if they are potentially exposed to it.
When users don't understand what we are scanning for and if it is current, they become fearful and have doubts. If they can't get the answer to the questions they have, they either have to live with that fear or engage with GitLab field staff.
There have been several customers conversations along the lines of "I heard about CVE XYZ - am I protected against it?" that this is intended to help address. This has come from non-security engineers, such as PMs, release managers, or CXOs.
Proposal
Create a new page inside GitLab.com that allows viewers to:
See how many vulnerabilities are currently in the Vulnerability Database
This requirement was removed since it requires a large amount of context for the numbers to be meaningful beyond a vanity metric. We anticipate many users will not have this context when viewing the screen so it is not helpful. We will consider re-adding it in a new issue if we get feedback that the raw number would be helpful.
The last time the database was updated
What our current mean time-to-merge is
Search for a specific CVE and see if we currently scan for it.
Domain must be inside GitLab.com and not part of the Handbook
Note: This page does not need to be part of a self-hosted installation - there can be a single copy hosted on GitLab.com.
Documentation should be updated to reference the new page.
What does success look like, and how can we measure that?
Field staff can confidently answer customers that a given vulnerability they are concerned about is being scanned for. => Canvas field staff that they get value out of the page
Customers can directly access vulnerability information to understand what we scan for. => Measure page views
@tstadelhofer can you review & provide your thoughts? This is something we need to start working on to give our users more visibility into our vulnerability database.
@stkerr I like the idea of showing the recently added vulns. The challenge with current implementations, such as Can's is that it doesn't show the date the vulns were added. Meaning, I have no idea how recent those vulns are. I can't help but think we could easily overcome this, just pointing out how it doesn't completely answer the question. However, I think with a combination of Can's work (with dates added) and the Security Dashboard, we can help alleviate concerns that our customers might have.
That makes sense. I figured we'd have to adapt those existing implementation, rather than use them as drop-ins, and probably make some tweaks or exclusions along the way. Is doing this grooming & breakdown something the VR team can do or do we need to pull in some other folks for help?
@stkerr I think we should start with the VR team and if they need help, we can reach out at that point. I also know that @idawson put together a tool to tell you if you are vulnerable to a specific CVE, given a specific configuration. Perhaps we start with that as the base and then have a section that shows recent CVE updates? Just a thought...
Starting with the VR team sounds good to me. I don't think I know about Isaac's tool that you mention. If it addresses the 4 points in the requirements above though, I'm all for it!
Sure thing @stkerr!
@julianthome@d0c-s4vage@idawson Do you believe this is something that we could achieve for 13.0? If so, what do you believe the scope is? I prefer to use relative sizing. Please see below table for reference...
I think we could combine the work for this issue together with https://gitlab.com/groups/gitlab-org/secure/vulnerability-research/-/epics/8 because they are strongly related. As a rough estimate I would assign the weight 3 (in the context of this issue). I think we should be able to setup a static page for %13.0. Just added this as a talking point to the agenda for our next VR meeting.
I suppose besides showing all the information mentioned above, the page should also use our branding. Maybe we could use the templates that are also used to release GitLab blog posts.
@julianthome While the two pages are related, their delivery should not be tied together and one should not block the other's completion. In terms of priority, this is higher priority than &8 (closed).
To @julianthome's point, there is a lot of overlap between the two sets of data (CVEs and gemnasium-db advisories). I could see them being merged at some point if it doesn't happen for the MVC.
As a rough estimate I would assign the weight 3
That seems about right to me. There's not a lot of unknowns, and we have existing code to base the page generation off of.
@julianthome While the two pages are related, their delivery should not be tied together and one should not block the other's completion. In terms of priority, this is higher priority than &8 (closed).
Yes, that makes sense. I was just thinking that it may make sense to prepare a single docker image that ships the common functionality/templates we need for generating both landing pages (for the CVE advisories as well as the package advisories). We could use https://gitlab.com/gitlab-com/www-gitlab-com/-/tree/master/sites/blog as an inspiration to make sure that the landing pages are consistent with the GitLab branding. Due to the commonalities between the data-formats as well as the goal we'd like to achieve (w.r.t. https://gitlab.com/groups/gitlab-org/secure/vulnerability-research/-/epics/8), we could at least bundle the commonly used functionality into a single docker image. I think that could safe us some time and helps to make sure that there isn't too much deviation in terms of look&feel for both landing pages.
@stkerr
Please let me know how frontend can be involved for %13.0 scope. It seems we would be involved in standing up a static page as mentioned by @julianthome above. Let us know if you would like frontend representation in the next sync session.
@stkerr We'd like to understand the value that the proposal offers. With all of these, we are happy to provide this data, we would just like to understand the value, as it will provide context for us to give a better experience.
See how many vulnerabilities are currently in the Vulnerability Database
It's unclear what the value is that this will provide. I'll give an example... With Go we have just over 80 vulns in our db. Is that good or bad? Also worth noting is that at a previous company we had 100s of thousands of vulns that we covered but that didn't matter. The question that needs to be answered is, did it catch the issue? If not, then having 100s of thousands really didn't matter.
The last time the database was updated
This one is somewhat interesting in that a customer can see that we are consistently updating our db. However, beyond that, I am not sure of what a customer would hope to get out of this information. Is there something more (not that there has to be)?
What our current mean time-to-merge is
MTTM is interesting in that it kind of goes hand-in-hand with when the db was last updated. However, if we are consistently meeting our SLO, I am not sure of what the use case for this.
Search for a specific CVE and see if we currently scan for it.
This one is the one that I believe has value in that a customer can search to see if an issue they are concerned about is covered by GitLab.
We'd like to understand the value that the proposal offers.
Great point to raise! My hope is that the Problem to Solve section can help with this and I'll augment it after writing this reply.
A big part of the value in this issue is that it provides visibility to our end-users, which they don't have today. Because users don't know what's going on with respect to what we scan for, they become fearful, have doubts about their security against threats, and have to either live with the fear or engage synchronously with the GitLab field team.
The goal of this issue is to provide visibility into what we have in our DB and that it is current, from both a high-level view (e.g. generic stats) and a low-level view (e.g. specific CVE queries). This will help instill more confidence in our customers, allow them to self-service better, and save our field staff time when those questions do come up.
However, if we are consistently meeting our SLO, I am not sure of what the use case for this.
I think it's not clear to users that we have an SLO nor that they can find the dashboard that tracks it. By publishing this information, they can read more about it to understand what GitLab's goals are. For the page, we can link to info describing the SLO in greater detail if there is a lot of text we don't want to duplicate.
Let me know if that helps - if it's not clear on the why we're doing this, we should keep this thread going until we're all on the same page.
Thanks @stkerr! It helps with items 2 and 4 in the proposal. Still not sure that it answers 1 and 3 though. Is there anyway you can elaborate on them? I ask because I feel as though item 2 covers item 3, making item 3 not necessary. For item 1, I honestly don't know how value can be derived from knowing the number of CVEs. I'm fully willing to accept that I'm missing something though.
For 2 & 3, they are very similar but do address slightly different things. #2 (closed) shows that we have made recent update and that user can know they are not looking at a stale page. #3 (closed) is showing that the updates are recent with respect to the various feeds - otherwise, a user could ask if the update we made today is covering info that is 6 months old.
On point #1 (closed), thinking about it more, I think I agree with you. It does require a lot of context for the number to be meaningful, which users are likely not going to have. What do you think of striking it for now & iterating with just 2, 3, & 4, then revisiting #1 (closed) based on feedback we get?
Is this new page/site meant to replace links to https://deps.sec.gitlab.com/ currently present in the advisories?
If yes, it might be useful for offline environments, as we could ship it along with the Gemnasium DB somehow.
As to the landing pages being meant to replace links to https://deps.sec.gitlab.com - I'm not sure, but it makes sense. @stkerr, is that in the plan for the landing pages?
@plafoucriere@d0c-s4vage in terms of replacing deps.sec.gitlab.com, that wasn't the original intention of the page. That said, it makes sense to start talking about if they are duplicating each other and/or what each provides that we could merge together. As a starting point, we could likely add cross-links between the two pages. We also need to add some more GitLab branding to that Gemnasium page