Dependency List contains duplicates (npm project)

Summary

A npm project may have duplicates in its dependency list: a single package-lock.json file may have the same package version multiple times in its dependencies. This issue may affect other types of dependency files - to be investigated.

The problem comes from the Gemnasium analyzer and is visible in the generated Dependency Scanning report it generates. It may impact gemnasium-python and gemnasium-maven as well.

Steps to reproduce

Create a npm project with a package-lock.json file, and where the same package version appears multiple times in the dependency graph.

Example Project

https://gitlab.com/gitlab-org/security-products/tests/js-npm/blob/master/qa/expect/gl-dependency-scanning-report.json#L11322-11333

What is the current bug behavior?

This JSON object appears twice in the dependencies of package-lock.json:

        {
          "package": {
            "name": "yallist"
          },
          "version": "2.1.2"
        }

What is the expected correct behavior?

yallist@v2.1.2 should appear only once in the list.

Possible fixes

In this particular case the root cause is the way the dependencies list is generated in the npm parser, to be fixed in parser/npm/npm.go.

That being said, the problem may be generic and affects multiple parsers, in which case it's best to remove the dups when converting the dependencies returned by the parser, in convert/convert.go.

/cc @gonzoyumo