Gemnasium fails to parse yarn.lock with git URLs

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Summary

Gemnasium v5.7.3 fails to parse yarn.lock if a dependency is referenced by git URL.

See https://docs.npmjs.com/cli/v8/using-npm/package-spec/#git-urls

Steps to reproduce

• Create a yarn project that depends on aframe 1.6.0, which itself depends on dmarcos/three-bmfont-text#eed4878795be9b3e38cf6aec6b903f56acd1f695.
• Enable Dependency Scanning in that project.
• Wait for the Dependency Scanning CI job to finish.

Example Project

What is the current bug behavior?

Gemnasium fails with the following error:

cannot find yarn dependency three-bmfont-text@dmarcos/three-bmfont-text#eed4878795be9b3e38cf6aec6b903f56acd1f695

What is the expected correct behavior?

No error

Relevant logs and/or screenshots

Full error message:

[FATA] [Gemnasium] [2024-10-08T17:00:44Z] [/go/src/app/cmd/gemnasium/main.go:83] ▶ scanning file /tmp/app/yarn.lock: parsing file /tmp/app/yarn.lock: cannot find yarn dependency three-bmfont-text@dmarcos/three-bmfont-text#eed4878795be9b3e38cf6aec6b903f56acd1f695 required by aframe@1.6.0

Further details

See https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v5.7.3/scanner/parser/yarn/classic/classic.go#L98

When parsing the following block of yarn.lock, spec is "three-bmfont-text@github:dmarcos/three-bmfont-text#eed4878795be9b3e38cf6aec6b903f56acd1f695".

• Spec.Name is "three-bmfont-text".
• Spec.Requirement is "github:dmarcos/three-bmfont-text#eed4878795be9b3e38cf6aec6b903f56acd1f695".

"three-bmfont-text@github:dmarcos/three-bmfont-text#eed4878795be9b3e38cf6aec6b903f56acd1f695":
  version "3.0.0"
  resolved "https://codeload.github.com/dmarcos/three-bmfont-text/tar.gz/eed4878795be9b3e38cf6aec6b903f56acd1f695"
  dependencies:
    array-shuffle "^1.0.1"
    layout-bmfont-text "^1.2.0"
    nice-color-palettes "^3.0.0"
    quad-indices "^2.0.1"    

However, when parsing the dependencies of the following block, the corresponding spec is three-bmfont-text@dmarcos/three-bmfont-text#eed4878795be9b3e38cf6aec6b903f56acd1f695.

• Spec.Name is "three-bmfont-text".
• Spec.Requirement is "dmarcos/three-bmfont-text#eed4878795be9b3e38cf6aec6b903f56acd1f695".

There is no github: in Spec.Requirement in that case.

aframe@^1.5:
  version "1.6.0"
  resolved "https://registry.yarnpkg.com/aframe/-/aframe-1.6.0.tgz#7f17461b36e08f3548e23d6d6bf8fbc0386c586f"
  integrity sha512-+P1n2xKGZQbCNW4lTwfue9in2KmfAwYD/BZOU5uXKrJCTegPyUZZX/haJRR9Rb33ij+KPj3vFdwT5ALaucXTNA==
  dependencies:
    buffer "^6.0.3"
    debug "^4.3.4"
    deep-assign "^2.0.0"
    load-bmfont "^1.2.3"
    super-animejs "^3.1.0"
    three "npm:super-three@0.164.0"
    three-bmfont-text dmarcos/three-bmfont-text#eed4878795be9b3e38cf6aec6b903f56acd1f695
    webvr-polyfill "^0.10.12"

As a result, the parser fails to find a package matching the spec.

Workaround

Add a before_script that alters yarn.lock and removes github: so that dependencies match packages.

gemnasium-dependency_scanning:
  before_script:
    - sed -i 's/@github:/@/g' yarn.lock

See #499069 (comment 2158584199)

Possible fixes

• Remove github: from spec requirement (what comes after @) using strings.CutPrefix. This allows the spec to be compared with dependencies since github: is implied in that context.
  • Update gemnasium and release new version.
  • Update depenency-scanning and release a new version.

/cc @cmutua @hacks4oats