Design: Warn about failed jobs in the Project Security Dashboard
Details
The Security Dashboards are based on the latest successful pipeline in the Default Branch. There are some cases when results can be incomplete in the dashboard, because one of the security jobs failed in this pipeline. Yet, as jobs are allowed to fail, the pipeline is still considered as successful.
Problem to solve
Users can't see easily if some data are missing from the dashboard because a job failed.
Problem statements
- How might we communicate the state of the vulnerability report [security dashboard] to users?
- How might we communicate the report is incomplete or missing data from a defective scan(s).
- How might we communicate the report is not current and should be updated?
Jobs to be done
Location: Project > Security Dashboard > Vulnerability List
Experience: Reviewing and prioritizing vulnerabilities
Primary - scoped to the core experience
When I am managing vulnerabilities for my organization, I want to address all serious and time-sensitive threats first, so I can ensure my company is not at risk of an imminent attack or breach.
Secondary - scoped to the problem
When I am preparing to triage vulnerabilities, I want to know if my report is current and includes the results from all of my enabled scanners, so I can feel confident I am viewing all detectable vulnerabilities in my project.
Tertiary - scoped to the problem/solution
When my report is not current or missing data, I want to identify the problem and begin the process of addressing it, So I can be assured I am prioritizing and triaging all detectable vulnerabilities in my project.
Intended users
Solution space & Scope
How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey
- Users should be able to identify if there is a problem with the state of their report
- Users should be able to determine the cause of the problem if their report is inaccurate or incomplete
Proposal
- Add description under page title:
The Security Dashboard shows the results of the last successful pipeline run on the default branch.
- Add pipeline widget with
Last updated
copy, how long ago the most recent pipeline passed, and a pipeline number which links to the pipeline page - Show failed job(s) badge, if any, which links to the failed jobs page of the pipeline
If jobs all passed successfully, no badge appears:
If job(s) failed, show badge with count of failures. Badge links directly to Failed jobs
tab of pipeline page:
- I'm not sure what the hover state of this component is in gitlab-ui, but there should be some indication that it can be clicked on, including a cursor change to the "pointer hand", as well as maybe an underline treatment of the text, e.g.:
Permissions and Security
Nothing new.
Documentation
Update https://docs.gitlab.com/ee/user/application_security/security_dashboard/ accordingly.
Availability & Testing
Have a pipeline with a failing job, check that we report it in the dashboard.
SET to add an end to end test as tracked at gitlab-org/quality/testcases#1005 (closed)
What does success look like, and how can we measure that?
UX:
- Pipeline widget appears at all times on the project level security dashboard
-
Last updated
time is updated after a pipeline runs on the default branch, includes the number of the pipeline which links to that pipeline page - If any security jobs fail, a red badge appears with the number of errors. There should be a hover state over this badge (see design above under
Proposal
). This badge links to theFailed jobs
tab of the pipeline.
What is the type of buyer?
Links / references
/cc @andyvolpe