Skip to content

Design: Warn about failed jobs in the Project Security Dashboard

Details

The Security Dashboards are based on the latest successful pipeline in the Default Branch. There are some cases when results can be incomplete in the dashboard, because one of the security jobs failed in this pipeline. Yet, as jobs are allowed to fail, the pipeline is still considered as successful.

Problem to solve

Users can't see easily if some data are missing from the dashboard because a job failed.

Problem statements

  • How might we communicate the state of the vulnerability report [security dashboard] to users?
    • How might we communicate the report is incomplete or missing data from a defective scan(s).
    • How might we communicate the report is not current and should be updated?

Jobs to be done

Location: Project > Security Dashboard > Vulnerability List
Experience: Reviewing and prioritizing vulnerabilities

Primary - scoped to the core experience

When I am managing vulnerabilities for my organization, I want to address all serious and time-sensitive threats first, so I can ensure my company is not at risk of an imminent attack or breach.

Secondary - scoped to the problem

When I am preparing to triage vulnerabilities, I want to know if my report is current and includes the results from all of my enabled scanners, so I can feel confident I am viewing all detectable vulnerabilities in my project.

Tertiary - scoped to the problem/solution

When my report is not current or missing data, I want to identify the problem and begin the process of addressing it, So I can be assured I am prioritizing and triaging all detectable vulnerabilities in my project.

Intended users

Solution space & Scope

How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey

  1. Users should be able to identify if there is a problem with the state of their report
  2. Users should be able to determine the cause of the problem if their report is inaccurate or incomplete

Proposal

  • Add description under page title:

The Security Dashboard shows the results of the last successful pipeline run on the default branch.

  • Add pipeline widget with Last updated copy, how long ago the most recent pipeline passed, and a pipeline number which links to the pipeline page
  • Show failed job(s) badge, if any, which links to the failed jobs page of the pipeline

If jobs all passed successfully, no badge appears:

image

If job(s) failed, show badge with count of failures. Badge links directly to Failed jobs tab of pipeline page:

image

  • I'm not sure what the hover state of this component is in gitlab-ui, but there should be some indication that it can be clicked on, including a cursor change to the "pointer hand", as well as maybe an underline treatment of the text, e.g.:

image

Permissions and Security

Nothing new.

Documentation

Update https://docs.gitlab.com/ee/user/application_security/security_dashboard/ accordingly.

Availability & Testing

Have a pipeline with a failing job, check that we report it in the dashboard.

SET to add an end to end test as tracked at gitlab-org/quality/testcases#1005 (closed)

What does success look like, and how can we measure that?

UX:

  • Pipeline widget appears at all times on the project level security dashboard
  • Last updated time is updated after a pipeline runs on the default branch, includes the number of the pipeline which links to that pipeline page
  • If any security jobs fail, a red badge appears with the number of errors. There should be a hover state over this badge (see design above under Proposal). This badge links to the Failed jobs tab of the pipeline.

What is the type of buyer?

GitLab Ultimate

Links / references

/cc @andyvolpe

Edited by Becka Lippert