Create Vulnerability Management supply chain investigation runbook

---

title: "Software Supply Chain Vulnerability Investigations"

description: "Provides actionable guidance for Vuln Mgmt team members to perform software supply chain investigations"

---

## Overview

The Vulnerability Management team will be asked to audit GitLab's software supply chain for evidence of compromised or otherwise vulnerable dependencies or tooling. Typically we are engaged via the `#security_discuss` Slack channel or via SIRT and/or PSIRT for this work.

## Creating an Investigation Issue

Once we are engaged we must ensure we have a confidential tracking issue in the [tracker project](https://gitlab.com/gitlab-com/gl-security/product-security/vulnerability-management/vulnerability-management-internal/vulnerability-management-tracker/-/issues) tracking the work, with the following details:

- A link to the advisory, blog post and any other sources of information relevant to the investigation

- A table with the following details:

  - Name of the affected component/package

  - Version(s) of the component that are affected

  - If the component appears in the Dependency List of any of the following project groups:

    - In GitLab the product itself? If so, are we dependant on an affected version?

    - In any projects under the `gitlab-org` group? If so, are we dependant on an affected version?

    - In any projects under the `gitlab-com` group? If so, are we dependant on an affected version?

- Recommendations on any needed follow-up and/or action based on severity, if known

- If the package is still available for install (:warning:) or has been pulled from the package registry (:white_check_mark:)

## Tools

### GitLab Dependency List / SBOM

The Dependency List is a feature of GitLab, available from the `Secure > Dependency List` menu for any project or group. When using the Dependency List, search for components by first selecting `Component =` in the search bar, then start typing the name of the component. If there are no results, the component is not a dependency; if the name does appear, select the component from the list, click out of the list, then run the search to get a list of affected projects within the group.

GraphQL APIs can also be used to query the Dependency List. For example a GraphQL query like the following to check if we have these packages in any projects under `gitlab-org`:

```json

{

  group(fullPath: "gitlab-org") {

    dependencies(componentNames: ["golang.org/x/crypto", "golang.org/x/net"]) {

      nodes {

        id

        name

        componentVersion {

          version

        }

        packager

        location {

          path

          blobPath

        }

      }

    }

  }

}

```

The [GraphQL Explorer](https://gitlab.com/-/graphql-explorer) can be used to run GraphQL queries against GitLab projects when logged in.

### Package Query

[Package Query](https://gitlab.com/gitlab-com/gl-security/product-security/vulnerability-management/vulnerability-management-internal/package-query) is a small tool that can be utilized for quickly querying the GitLab Dependency List data.

Package Query queries can be stored in a JSON file and look like this:

```json

{

  "timestamp": "2025-12-04T10:33:54+08:00",

  "groupPath": "gitlab-org",

  "packagesQueried": [

    "golang.org/x/crypto",

    "golang.org/x/net"

  ]

}

```

### GitLab Gem Database

For investigating Ruby gems, the [Gem Database](https://gem-source-database-1392d3.gitlab.io/) is useful to determine which gems are actually installed and used in different releases of GitLab. Not every gem listed in a `lock` file is necessarily installed.

## Coordinating With Teams

Based on this initial information, Vulnerability Management can work with other Security teams to prioritize further investigation and mitigation. Where impact is identified, we should ensure SIRT & PSIRT are engaged so we can treat the remediation and mitigation work as an incident, and work with engineering and release management to ship updated packages if necessary. PSIRT can be tagged on the issue with `@gitlab-com/gl-security/product-security/appsec/psirt-group`. See the [PSIRT Case Lifecycle](../../psirt/runbooks/psirt-case-lifecycle.md) handbook page for more information.

The Vulnerability Management team member on weekly triage rotation is not expected to perform all analysis, remediation, and mitigation work. The relevant teams should be properly engaged so we can scale to respond and communicate impact.

## Media and Customer Inquiries

Often supply-chain related vulnerabilities are published in the media and cause concern for customers about whether or not GitLab is impacted - it is important that as part of any supply chain investigation we work with PSIRT to drive proactive communication with customers to reassure them that we are either aware and investigating, have investigated and found no impact, or have determined impact and require specific actions from customers, including actions we have taken for GitLab's hosted offerings. Whilst we do not typically drive this process, it's important that we ensure that we engage with it to provide timely and accurate impact determinations for communication purposes.

## Specific Guidance

### POA&M

TODO

### SLA Exceptions

As part of the weekly triage rotation the Vulnerability management team reviews & approves/denies SLA Exception requests, SLA Exception Requests can be raised by any other Gitlabber by following the [SLA Exception Guidelines](../sla-exceptions/). These steps are here to guide the review process and provide consistent templated Approved/Denied messaging to the SLA Exception requester.

      - Detail why the existing extension was missed.

      - A requested to update the exception length if need be.

    note: An extension to the initial exception length is not a given and will require a approval by the Vulnerability Management Team.

    note: An extension to the initial exception length is not a given and will require approval by the Vulnerability Management Team.

    If all remediation work has been completed and deployed

      - Please confirm this is the case.

    The Vulnerability Management team

    ```

### Software Supply Chain Vulnerability Investigations

PSIRT or other teams may reach out about news of a new vulnerability or malware attack that may impact the software supply chain. A potential software supply chain vulnerability should be treated with high priority.

The team member on weekly triage rotation should take point on creating an initial supply chain vulnerability investigation issue, using the template available in the Vulnerability Management Internal project. Note a link to this investigation issue in the weekly triage issue for tracking.

The team member on weekly triage rotation is not expected to perform all analysis, remediation, and mitigation work. The relevant teams should be properly engaged so we can scale to respond and communicate impact.

Please see the [Software Supply Chain Vulnerability Investigations runbook](supply-chain-investigations.md) for details.

### POA&M

TODO

### Deviation Requests

TODO