Investigate requiring specific fixes to integrate with vulnerable code sample

Problem

Having an LLM suggest a remediation for potentially vulnerable code is inherently risky as it may suggest an incomplete or incorrect fix.

Proposal

Consider re-engineering prompts to include a hardcoded set of fixes per vulnerability type and language. The prompt would instruct the LLM to merge the vulnerable code with the suggested fix that has been verified by GitLab to actually work.

Example:

System Prompt: You are a developer given an example vulnerable function along with a suggested fix. Apply the suggested fix but rewrite the code in the <code> tag to use the fix logic.

Prompt: Given the following vulnerable code included in the <code> tag:

<code>
func Input(userInput str) {
    f, err := os.ReadFile(userInput)
}
</code>

Require that the `userInput` be restricted to a protected directory such as `/tmp/`. Rewrite the above function to use the below code included in the <fix> tag to match the input variable names and functions:

<fix>
	// restrict all file access to this path
        const basePath = "/tmp/"
        // resolve the full path, but only use our random generated id
        resolvedPath, err := filepath.Join(basePath, filepath.Base(data.id))
        if err != nil {
            log.Fatal(err)
        }
        // verify the path is prefixed with our basePath
        if !strings.HasPrefix(resolvedPath, basePath) {
            log.Fatal("path does not start with basePath")
        }
        // process / work with file
</fix>

Example output from Claude-3-opus:

Here is the rewritten function using the suggested fix to restrict file access to the /tmp/ directory:

<code> 
func Input(userInput string) { 
    // restrict all file access to this path 
    const basePath = "/tmp/" // resolve the full path, but only use the user provided input 
    resolvedPath, err := filepath.Join(basePath, filepath.Base(userInput)) 
    if err != nil { 
         log.Fatal(err) 
    } // verify the path is prefixed with our basePath 
    if !strings.HasPrefix(resolvedPath, basePath) { 
         log.Fatal("path does not start with basePath") 
    } 
    // process / work with file 
    f, err := os.ReadFile(resolvedPath) 
    if err != nil { 
        log.Fatal(err) 
    } 
    // rest of function logic... 
} 
</code>

Auto-Summary 🤖

Discoto Usage

---

Points

Discussion points are declared by headings, list items, and single lines that start with the text (case-insensitive) point:. For example, the following are all valid points:

• #### POINT: This is a point
• * point: This is a point
• + Point: This is a point
• - pOINT: This is a point
• point: This is a **point**

Note that any markdown used in the point text will also be propagated into the topic summaries.

Topics

Topics can be stand-alone and contained within an issuable (epic, issue, MR), or can be inline.

Inline topics are defined by creating a new thread (discussion) where the first line of the first comment is a heading that starts with (case-insensitive) topic:. For example, the following are all valid topics:

• # Topic: Inline discussion topic 1
• ## TOPIC: **{+A Green, bolded topic+}**
• ### tOpIc: Another topic

Quick Actions

Action	Description
/discuss sub-topic TITLE	Create an issue for a sub-topic. Does not work in epics
/discuss link ISSUABLE-LINK	Link an issuable as a child of this discussion

---

Last updated by this job

Discoto Settings

---
summary:
  max_items: -1
  sort_by: created
  sort_direction: ascending

See the settings schema for details.