Skip to content

UX Audit: SAST configuration UI

⚡️ Summary

As part of Secure: Scan Configuration Evaluation, we are performing an audit of the states and UI patterns for every security scanner with a configuration interface. The goal of the audit is to identify patterns and inconsistencies and create recommendations focused on improving consistency and learnability between the scanners. This issue is to track and document findings for SAST configuration.

The parent issue that documents all scanners being audited is linked below:
👉 #340334 (closed)

📋 Plan

  • Identify relevant JTBD
  • Evaluate the configuration process and document existing workflows (user flows)
  • Capture screenshots of the configuration interface including all possible states
  • Review any new or upcoming configuration changes for SAST and note anything that deviates from the audit findings and/or adjust findings accordingly
  • Document findings in an easy to digest way

💼 JTBD

  1. When I am configuring a CI/CD security scan, I want to specify which assets need to be scanned and under which circumstances, So that I can ensure my assets are secure prior to or at their release.
  2. When I am configuring a security scan, I want to specify which types of vulnerabilities the scan should detect, So that we don't waste time sorting through irrelevant findings.
  3. When I am either enabling or configuring a security scan, I want to run a demo scan, So that I can validate my configuration before it is implemented

📸 Screenshots

See design section below

🚶 Workflow

🎨 FigJam File image image

📄 Relevant issues

  1. Configuration UI for SAST (epic)
    1. Show a generic static message in the SAST Config UI
    2. Display per-variable help link in SAST Config UI
    3. SAST Config input validation
    4. Enhancing SAST Config UI Elements
  2. Static Analysis UX (epic)
    1. Add repository view to Excluded Paths variable in SAST Config UI
    2. Update SAST Configuration UI collapse/ accordion component (minor)
  3. Custom rulesets

💡 Findings

Does the SAST configuration UI address the JTBD?

2 of 3 JTBD are at least partly addressed within the SAST configuration UI

  1. ✓ JTBD 1 is addressed by the Excluded Paths field
  2. ✓ JTBD 1 is partly addressed by the SAST Analyzers section of the configuration UI. Users can select which languages to scan but cannot choose which vulnerabilities get reported
  3. 𐄂 JTBD 3 is not directly addressed within the SAST configuration flow. There is no way to run a "demo" scan to validate the configuration from the UI.

Findings unique to SAST

  1. The configuration process for SAST differs between the free and ultimate plan tiers. The free tier does not have access to the UI configuration screen. GitLab core users (free tier) can enable SAST from the UI via an auto-generated merge request (similar to secret detection and dependency scanning).
  2. SAST configuration is positioned as one of the primary entry points into application security
    • Promoted via a banner on project home page
    • "Add security testing" button on the project homepage links to SAST documentation
  3. The configuration process is relatively simple compared to dynamic analysis scanners, but more complex than "one-click configuration" scanners like dependency scanning and secret detection
    • Some configuration options are available to users
    • The .gitlab-ci.yml file is automatically created and/or updated for users when enabling SAST (Note: this behavior is shared with dependency scanning and secret detection)
    • The current/active configuration values are maintained in the UI when SAST is enabled
    • Users are highly encouraged not to edit the SAST configuration. This is communicated through a few different techniques:
      • The page description mentions: Configuration changes made here override those provided by GitLab and are excluded from updates
      • Info text (a "description") is displayed under any input fields that are changed from their default state: Using custom settings. You won't receive automatic updates on this variable. Restore to default
      • Info banner is displayed if any analyzer is disabled (view screenshot)

Findings & themes observed across multiple scanners

  1. The end of the configuration process (merging code to enable the feature) doesn't do a great job assuring users that the scanner is properly enabled
    • Enabling SAST requires a merge request and users who enable a scanner may not have access to merge the change themselves
    • When SAST is successfully enabled, the only place to confirm this is on the pipeline details page or within the Security & Compliance menu. The change in system status is not easily discovered.
  2. Field validation is minimal or non-existing:
    1. There is no form field validation within the SAST configuration UI, but fields can be reset to default if they are altered (see related issue about validation)
Edited by Michael Fangman