Dependency Scanning failed for sbt project with error "panic: simple: adding self edge [recovered]"
<!---
Please read this!
Before opening a new issue, make sure to search for keywords in the issues
filtered by the "regression" or "bug" label:
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression
- https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=bug
and verify the issue you're about to submit isn't a duplicate.
--->
### Summary
<!-- Summarize the bug encountered concisely. -->
Using dependency scanning template for a sbt project failed with error "panic: simple: adding self edge [recovered]"
```yml
include:
- template: Dependency-Scanning.gitlab-ci.yml
variables:
SECURE_LOG_LEVEL: debug
```
A similar error might happen when processing some yarn projects. In that case the error originates form `convert.NewGraph`, when generating the Dependency Scanning report.
### Further details
The gemnasium-maven analyzer [runs the `dependencyDot` task](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/builder/sbt/sbt.go#L79) of the [sbt-dependency-graph plugin](https://github.com/sbt/sbt-dependency-graph) to export dependency graphs of Sbt projects to DOT files, and then parses DOT files [using gonum](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/scanner/parser/sbt/dotgraph.go#L18). More specifically, the DOT file parsers creates a `gonum/graph/simple.DirectedGraph`, and then decode it using `gonum/graph/encoding/dot`. See https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/scanner/parser/sbt/dotgraph.go#L18
Unfortunately, [DirectedGraph.SetEdge](https://pkg.go.dev/gonum.org/v1/gonum/graph/simple#DirectedGraph.SetEdge) doesn't support self edges:
> SetEdge adds e, an edge from one node to another. If the nodes do not exist, they are added and are set to the nodes of the edge otherwise. It will panic if the IDs of the e.From and e.To are equal.
### Proposal
Define `dotGraph.SetEdge` so that it does nothing when the argument is a self edge. If not, then it simply delegates to the `SetEdge` method of the embedded directed graph.
```go
func (g *dotGraph) SetEdge(e graph.Edge) {
if e.From() == e.To() {
return
}
g.DirectedGraph.SetEdge(e)
}
```
See https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/scanner/parser/sbt/dotgraph.go#L11
```go
// dotGraph is the simple.DirectedGraph representation of a dot file
type dotGraph struct {
*simple.DirectedGraph
}
```
Similarly, `convert.NewGraph` must be updated in order to skip self-edges when building the dependency graph.
An alternative would be to use a another type of graph that supports self edges, but we could have problems when processing these self edges further down the road in the Gemnasium analyzer.
In any case, we should investigate and figure out what causes these self edges, and what are the implications for features that rely on the dependency graph. See https://gitlab.com/gitlab-org/gitlab/-/issues/333254#note_618859518
### Implementation plan
- [ ] Update file parser in `gemnasium` project.
- Change `sbt` file parser, to skip self-edges when parsing the DOT graph.
- Change `convert.NewGraph`, to skip self-edges when building the graph.
- Release a new PATCH version.
- [ ] Update `gemnasium-maven`.
- Update `gemnasium` git submodule.
- Release a new PATCH version.
gemnasium-python doesn't seem to be impacted by this, but it might be safer to release a new version anyway, after upgrading to the latest version of gemnasium (git submodule).
### Steps to reproduce
<!-- Describe how one can reproduce the issue - this is very important. Please use an ordered list. -->
1. sbt-dependency-graph and sbt-dependency-check settings are configured in plugins.sbt and build.sbt (for local usage)
2. Include dependency scanning template
3. Run the pipeline
### Example Project
<!-- If possible, please create an example project here on GitLab.com that exhibits the problematic
behavior, and link to it here in the bug report. If you are using an older version of GitLab, this
will also determine whether the bug is fixed in a more recent version. -->
### What is the current *bug* behavior?
<!-- Describe what actually happens. -->
Job `gemnasium-maven-dependency_scanning` failed with error "panic: simple: adding self edge [recovered]". No `gl-dependency-scanning-report.json` file is generated
### What is the expected *correct* behavior?
<!-- Describe what you should see instead. -->
Job `gemnasium-maven-dependency_scanning` is expected to succeed with `gl-dependency-scanning-report.json` file generated
### Relevant logs and/or screenshots
<!-- Paste any relevant logs - please use code blocks (```) to format console output, logs, and code
as it's tough to read otherwise. -->
Job console log
```
[info] welcome to sbt 1.4.3 (AdoptOpenJDK Java 11.0.7)
[info] loading settings for project global-plugins from plugins.sbt ...
[info] loading global plugins from /root/.sbt/1.0/plugins
[info] loading settings for project map-update-pipeline-build from assembly.sbt,plugins.sbt ...
[info] loading project definition from /builds/ce/content-management/content-sourcing/map-update-pipeline/project
[info] loading settings for project root from build.sbt ...
[info] set current project to Map Updating Pipeline (in build file:/builds/ce/content-management/content-sourcing/map-update-pipeline/)
[warn] there's a key that's not used by any other settings/tasks:
[warn]
[warn] * root / dependencyCheck / logLevel
[warn] +- /builds/ce/content-management/content-sourcing/map-update-pipeline/build.sbt:10
[warn]
[warn] note: a setting might still be used by a command; to exclude a key from this `lintUnused` check
[warn] either append it to `Global / excludeLintKeys` or call .withRank(KeyRanks.Invisible) on the key
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
[info] Wrote dependency graph to '/builds/ce/content-management/content-sourcing/map-update-pipeline/target/dependencies-compile.dot'
[success] Total time: 3 s, completed Jun 10, 2021, 11:05:33 AM
[DEBU] [gemnasium-maven] [2021-06-10T11:05:34Z] ▶ Exporting dependencies for /builds/ce/content-management/content-sourcing/map-update-pipeline/lambdaLayer/build.sbt
[DEBU] [gemnasium-maven] [2021-06-10T11:05:54Z] ▶ /opt/asdf/shims/sbt dependencyDot
[info] Loading settings for project global-plugins from plugins.sbt ...
[info] Loading global plugins from /root/.sbt/1.0/plugins
[info] Loading settings for project lambdalayer-build from plugins.sbt ...
[info] Loading project definition from /builds/ce/content-management/content-sourcing/map-update-pipeline/lambdaLayer/project
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
[info] Loading settings for project root from build.sbt ...
[info] Set current project to root (in build file:/builds/ce/content-management/content-sourcing/map-update-pipeline/lambdaLayer/)
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
[info] Wrote dependency graph to '/builds/ce/content-management/content-sourcing/map-update-pipeline/lambdaLayer/target/dependencies-compile.dot'
[success] Total time: 1 s, completed Jun 10, 2021, 11:05:54 AM
panic: simple: adding self edge [recovered]
panic: simple: adding self edge
goroutine 1 [running]:
gonum.org/v1/gonum/graph/encoding/dot.copyGraph.func1(0xc000220a70)
/go/pkg/mod/gonum.org/v1/gonum@v0.8.1/graph/encoding/dot/decode.go:87 +0xb0
panic(0x83b2a0, 0x922c30)
/usr/local/go/src/runtime/panic.go:969 +0x1b9
gonum.org/v1/gonum/graph/simple.(*DirectedGraph).SetEdge(0xc0004ca270, 0x931f60, 0xc0001f11e0)
/go/pkg/mod/gonum.org/v1/gonum@v0.8.1/graph/simple/directed.go:189 +0x4ed
gonum.org/v1/gonum/graph/encoding/dot.(*simpleGraph).addEdgeStmt(0xc0002209c8, 0x935520, 0xc000550028, 0xc000193080)
/go/pkg/mod/gonum.org/v1/gonum@v0.8.1/graph/encoding/dot/decode.go:281 +0x1fd
gonum.org/v1/gonum/graph/encoding/dot.(*simpleGraph).addStmt(0xc0002209c8, 0x935520, 0xc000550028, 0x92fd20, 0xc000193080)
/go/pkg/mod/gonum.org/v1/gonum@v0.8.1/graph/encoding/dot/decode.go:197 +0x185
gonum.org/v1/gonum/graph/encoding/dot.copyGraph(0x935520, 0xc000550028, 0xc000193a70, 0x0, 0x0)
/go/pkg/mod/gonum.org/v1/gonum@v0.8.1/graph/encoding/dot/decode.go:103 +0x199
gonum.org/v1/gonum/graph/encoding/dot.Unmarshal(0xc00013e000, 0xc75d, 0xfe00, 0x935520, 0xc000550028, 0x5, 0xc00013e000)
/go/pkg/mod/gonum.org/v1/gonum@v0.8.1/graph/encoding/dot/decode.go:53 +0x95
gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2/scanner/parser/sbt.newDotGraph(0xc00013e000, 0xc75d, 0xfe00, 0xc00013e000, 0xc75d, 0xfe00)
/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2@v2.29.6/scanner/parser/sbt/dotgraph.go:18 +0x132
gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2/scanner/parser/sbt.Parse(0x92e540, 0xc000550020, 0xc00017e2d8, 0xc00017e388, 0x852900, 0xc000101890, 0xc00017e2c0, 0xc00017e2c0, 0x0, 0x0)
/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2@v2.29.6/scanner/parser/sbt/sbt.go:63 +0x8e
gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2/scanner.parseReader(0x92e540, 0xc000550020, 0x8ca468, 0x8af253, 0x3, 0x8b005d, 0x5, 0xc00006b3b0, 0x1, 0x1, ...)
/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2@v2.29.6/scanner/scanner.go:163 +0x55
gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2/scanner.Scanner.scanReader(0xc00001c378, 0xd, 0xc00001a258, 0x40, 0xc00001a0d5, 0x3c, 0xc0000182b6, 0x6, 0x92e540, 0xc000550020, ...)
/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2@v2.29.6/scanner/scanner.go:150 +0xf9
gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2/scanner.Scanner.ScanFile(0xc00001c378, 0xd, 0xc00001a258, 0x40, 0xc00001a0d5, 0x3c, 0xc0000182b6, 0x6, 0xc0001fe000, 0x67, ...)
/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2@v2.29.6/scanner/scanner.go:142 +0x18d
gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2/scanner.Scanner.ScanProjects(0xc00001c378, 0xd, 0xc00001a258, 0x40, 0xc00001a0d5, 0x3c, 0xc0000182b6, 0x6, 0xc00001e8c0, 0x42, ...)
/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/gemnasium/v2@v2.29.6/scanner/scanner.go:118 +0x3ea
main.analyze(0xc00003af80, 0xc00001e8c0, 0x42, 0x1, 0x1, 0x0, 0x1)
/go/src/app/analyze.go:102 +0x3f7
gitlab.com/gitlab-org/security-products/analyzers/command.Run.func1(0xc00003af80, 0x0, 0x0)
/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/command@v1.1.0/run.go:137 +0x559
github.com/urfave/cli/v2.(*Command).Run(0xc000136480, 0xc00003aa00, 0x0, 0x0)
/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/command.go:163 +0x4ed
github.com/urfave/cli/v2.(*App).RunContext(0xc000105380, 0x932be0, 0xc000020110, 0xc00000c080, 0x2, 0x2, 0x0, 0x0)
/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:313 +0x81f
github.com/urfave/cli/v2.(*App).Run(...)
/go/pkg/mod/github.com/urfave/cli/v2@v2.3.0/app.go:224
main.main()
/go/src/app/main.go:36 +0x379
```
### Output of checks
<!-- If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com -->
#### Results of GitLab environment info
<!-- Input any relevant GitLab environment information if needed. -->
<details>
<summary>Expand for output related to GitLab environment info</summary>
<pre>
(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)
(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
</pre>
</details>
issue