Smoke Test for Vulnerability Management

🔥 Smoke Tests 🚒

In addition to automated testing and the items outlined in the issues in GA Production Readiness (&14350 - closed), let's do some manual smoke tests! This will help to ensure we catch any possible edge cases early and confirm that everything is functioning as expected before the GA rollout.

Scenarios

  • A: We can successfully disable the feature if needed with FF
  • B: Ensure the AI features is only accessible for Ultimate users with a Developer role or higher
  • C: Test functionalities in SaaS environment
  • D: Test when each FF is enabled/disabled
  • E: The buttons for both AI features don't show on non-SAST vulnerabilities
  • F: The buttons show for SAST vulnerabilities
  • G: Experimental and GA features show for projects where the FF is disabled
  • H: Only displayed with Use GitLab Duo features settings
  • I: Independent of Use experiment and beta Duo features settings
  • J: Cloud connector check for self managed customers

Setup

  1. Clone this demo repo to run a pipeline and generate a vulnerability report > https://gitlab.com/gitlab-examples/security/security-reports

How to Test

  1. Create a separate thread for each scenario:

Example:

# A: We can successfully disable the feature if needed with FF
  1. Respond to that thread with your testing result:
Verified: [ENVIRONMENT] | [LINK, if suitable]

| [YOUR TEST] | Outcome |
| ------ | ------ |
|  [SCREENSHOT or text]      |   [SCREENSHOT]     |
|        |        |
Edited by Samantha Ming