Summit 2024 Code Challenge implementation details

Summary

  • Description of challenge: #443835 (closed)
  • The Google slides are available for internal team members to review and give a high-level overview.
  • The code written for the challenge is all on this branch, but it's messy there so can be a bit difficult to sift through.
  • Example Java project with a vulnerability is here.
  • This issue describes how the team solved the challenge in more (technical) detail and with links.

Vulnerabilities database is updated based on a mailing list message

  • No code required to complete this piece of the code challenge, just configuration of existing features.
  • We configure Continuous Vulnerability Scanning for dependencies.
  • We do not need to parse a mailing list message because, when continuous vulnerability scanning is enabled, GitLab automatically adds new vulnerabilities to the advisory database.
  • When a vulnerability is created in the advisory database, that is shown in the vulnerability report for any project that depends on an affected component.

GitLab flags all the projects using the vulnerable library

  • No code required to complete this piece of the code challenge, just configuration of existing features.
  • Normally, a new pipeline is created with each merge to the main branch for the project. To ensure that new vulnerabilities are flagged via GitLab CI even while the team is sleeping, scheduled pipelines are used to ensure that a new pipeline, including dependency scanning, is done hourly (or at whatever cadence is desired).
  • Using GitLab's existing security policy project feature, a group owner is able to ensure that all projects in their group or instance run dependency scanning as part of each GitLab CI run.

Duo AI suggests a fix for the vulnerability, the fix being updating the library

  • The scheduled pipeline includes security scanning jobs (as part of the enforced scan execution policy for the project).
  • The security scanning job runs a script that takes the vulnerability report for the project (generated by GitLab) and compares it with the CISA.gov Known Exploited Vulnerabilities Catalog (KEV catalog).
    • For the Code Challenge, we added this script directly to the example project. If this were to become part of the GitLab product, adding it as a component to the CI/CD catalog would be better.
    • For the Code Challenge, we took a short-cut and just downloaded the vulnerability data in JSON Format from CISA.gov and used the downloaded file for this comparison. In production, you'd want to use an API to get up-to-date results.
  • If a vulnerability in the project matches a vulnerability in the KEV catalog with a score of "High" or "Critical" (7 or higher), that means it is a high priority vulnerability to mitigate. As a result, a match will kick off an automated process for remediating the vulnerability.

The fix is added to a merge request that is automatically merged after tests pass

NOTE: Several years ago, groupcomposition analysis built an MVC for feature called Auto-Fix or Auto-Remediation. That feature automatically created MRs for suggested vulnerability fixes. The docs for that MVC were never merged and the feature is behind a disabled feature flag. The code for this feature is slated to be removed entirely. The discovery for that feature also looked into auto-merging the auto-fix MR. It would make sense to coordinate with the members of that team to understand why the feature was never rolled out before proceeding with any work to re-implement this feature.

  • The housekeeper gem, developed by GitLab, runs the "upgrade dependency" keep.
    • The upgrade dependency keep makes an API call to Anthropic's Claude 3 Opus model. In that API call, the vulnerability JSON and dependency list file for the project are sent to Claude as additional context and the LLM prompt requests the upgraded dependency version to use for the project.
    • Housekeeper takes the dependency version suggested by Claude and uses it to run the upgrade command for the dependency.
    • Housekeeper opens an MR using the diff that results from the upgrade command
    • The MR is created using HOUSEKEEPER_GITLAB_API_TOKEN, which belongs to a Service Account with sufficient permissions on the target project.
    • Housekeeper may not be the best tool for doing these scans as part of the GitLab product. For an example of vuln scanning + MR creation logic that does not require Housekeeper, see this project.
  • For the Code Challenge, a temporary solution was to set auto-merge on the MR automatically so that it would merge, but this approach would not work for projects with stricter approval rules.
  • GitLab CD automatically deploys the application when the MR is merged into the main branch.

GitLab detects that the Kubernetes resource usage is increasing too much

The fix is reverted by deploying an old version of the application

  • When the metric exceeds a certain threshold, AutoFlow rolls back the deployment and automatically creates an issue

An issue is opened with the AI explaining the vulnerability, the fix that was attempted, and problem with the deployment

Edited by Jessie Young