Implementation: Add Explain this Vulnerability Button
Why are we doing this work
This implementation issue tracks a beta for explaining a vulnerability using OpenAI.
Requirements
Click a button on a vulnerability that auto-generates a ChatGPT prompt that:
- Explains a vulnerability
- Tells the user how they can resolve it
- Recommends what needs to be changed in the code
- The results land in a drawer
Scope
- This feature is available for SAST detected vulnerabilities only. Vulnerabilities must have a file and line number. Please see the Relevant links section below for experiment results.
Designs
Design issue: #407124 (closed)
GPT Prompt examples
MVC includes:
- Vulnerability name
- CWE
- Code block
example 1
example 2
Resources
- Other teams are implementing similar functionality. We should leverage what we can from their teams.
- AI Common integration - https://gitlab.com/groups/gitlab-org/-/epics/10221
-
groupsource code
- design at https://gitlab.com/gitlab-org/gitlab/-/issues/403728+
- implementation at https://gitlab.com/gitlab-org/gitlab/-/issues/403718+
- Webgoat project has a lot of real SAST vulnerabilities - https://gitlab.com/gitlab-org/security-products/tests/webgoat/-/security/vulnerability_report/?scanner=GitLab.SAST
- Webgoat.net (seems to work better locally) https://gitlab.com/gitlab-org/security-products/tests/webgoat.net
Relevant links
Non-functional requirements
-
Documentation: Not required for alpha features. -
Feature flag: Yes - this should be controlled by a feature flag, at least on the frontend. -
Performance: -
Testing:
Implementation plan
- Add a button to the vulnerability details page
- Note: This should only be available for SAST vulnerabilities with file information.
- Open drawer
- API passing vulnerability ID
- Render response
- Create API that takes in Vulnerability ID
- Integrate with AI abstraction layer
- Retrieve vulnerability code snippet from file contents
- Extract X lines surrounding the identified line number of the vulnerability
- Include temperature of 0.3 following other projects
- Pass vulnerability code snippet to OpenAI/GPT API
- requesting markdown format
- Return formatted response OR error message if there is a failure
Schedule to hit release on .com on 2023-04-19
- Thursday 2023-04-13
- Backend MR to review
- Malcom (APAC)
- Maintainer: Mehmet (EMEA)
- Backend MR to review
- Friday
- Base Backend MR merged
- Daniel to test against backend MR
- Mo to add remaining backend (revised prompt with file contents, markdown formatting)
- Daniel to provide recorded demo using the experimental API (internal users only)
- Monday
- F&F Day
- Tuesday
- Frontend MR into review
- Frontend MR merged
- Wednesday 2023-04-19
- Merged to production behind a feature flag.
- Enable on a test project.
- Enable on gitlab-org.
- Enable on other internal projects for internal users that are interested in testing this feature.
- Enable globally on .com at 5PM MST if all other testing goes well.
Verification steps
- .
Verification projects
Staging
- https://staging.gitlab.com/govern-team-test/verification-projects/webgoat.net
- Vulnerability: https://staging.gitlab.com/govern-team-test/verification-projects/webgoat.net/-/security/vulnerabilities/3613977
Production
- https://gitlab.com/gitlab-org/govern/threat-insights-demos/personal-test-projects/webgoat.net
- Vulnerability:https://gitlab.com/gitlab-org/govern/threat-insights-demos/personal-test-projects/webgoat.net/-/security/vulnerabilities/82203662
Tofa Verification Projects
Staging
- https://staging.gitlab.com/govern-team-test/verification-projects/webgoat.net.tofa
- Vulnerability: ttps://staging.gitlab.com/govern-team-test/verification-projects/webgoat.net.tofa/-/security/vulnerabilities/3639303
Production
- https://gitlab.com/gitlab-org/govern/threat-insights-demos/personal-test-projects/webgoat.net.tofa
- Vulnerability: https://gitlab.com/gitlab-org/govern/threat-insights-demos/personal-test-projects/webgoat.net.tofa/-/security/vulnerabilities/82664413
Sequence Diagram
sequenceDiagram
autoNumber
actor User
participant Banner as UI Banner
participant Drawer as UI Drawer
participant Abstraction as Abstraction Layer
participant API as AI API
User->>Banner: Click Try Button
Banner->>Drawer: Load Drawer
Drawer--)+Abstraction: Send Vulnerability ID
Abstraction->>Abstraction: Get Vulnerability Data
Abstraction->>Abstraction: Get File Contents
Abstraction->>Abstraction: Construct Prompt
Abstraction--)+API: Send Prompt
API->>-Abstraction: Return response
Abstraction->>-Drawer: Render AI response
Edited by Neil McCorrison