Skip to content

Dependency scanning on Golang 1.21 creates false positives

Summary

Go version 1.21 introduced a new "toolchain" directive to go.mod, but the go command included in the gemnasium container can't handle the toolchain directive, so it falls back to go.sum parsing, which has known issues.

Relevant links

  • Go 1.21 release notes

    Go 1.21 introduces a small change to the numbering of releases. In the past, we used Go 1.N to refer to both the overall Go language version and release family as well as the first release in that family.

  • Go Toolchains

    Before Go 1.21, the initial release of a Go toolchain was version 1.N, not 1.N.0, so for N < 21, the ordering is adjusted to place 1.N after the release candidates.

Steps to reproduce

Example Project

What is the current bug behavior?

What is the expected correct behavior?

Relevant logs and/or screenshots

[WARN] [Gemnasium] [2023-12-18T00:15:19Z] ▶ Non-fatal error encountered while building project: loading packages used by main module: listing go packages cmd: /usr/bin/go list -deps -test -e -json=ImportPath,Standard,Module ./... dir: /builds/gitlab-com/gl-security/threatmanagement/vulnerability-management/vulnerability-management-internal/vulnmapper stderr: go: errors parsing go.mod:
/builds/gitlab-com/gl-security/threatmanagement/vulnerability-management/vulnerability-management-internal/vulnmapper/go.mod:5: unknown directive: toolchain
: exit status 1

Output of checks

This bug happens on GitLab.com

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Possible fixes

Implementation plan

  • Add golang.org/x/mod/modfile as a dependency: go get golang.org/x/mod/modfile@latest
  • Add a parseGoMod that takes an input file name, reads the file, and parses with modfile.Parse. It should convert the modules to a list of parser.Package. See line 183 for an example of how to convert a module to a parser.Package. Deduplication isn't necessary for this case because the modfile.Parse function handles this for us.
  • Update the Go Builder so that it doesn't return early if listPackages returns an error. Instead, it should call parseGoMod, and write the file using return writeGoProjectModulesJSONFile(absOutputPath, modules).
  • Remove the go.sum specs and add some for go.mod.
  • Document go mod tidy requirement for go.mod files. See #436092 (comment 1905600124)
Edited by Oscar Tovar