Add package to manipulate gemnasium-db repo

What does this MR do?

Add a package to manipulate a git clone of gemnasium-db:

• add Advisory, used to decode YAML advisories
• add Repo, used to list YAML advisories

This MR does NOT cover:

• synchronizing local gemnasium-db directory with remote gemnasium-db repo
• adding gemnasium-db to the Docker image
• evaluating affected ranges

What are the relevant issue numbers?

gitlab-org/gitlab#14630 (closed)

changed milestone to %12.4

added Category:Dependency Scanning [DEPRECATED] backend backstage [DEPRECATED] devopssecure groupcomposition analysis labels

@adamcohen Could you review this even though it's WIP?

It's WIP because it's to be rebased on top of !27 (merged), then used instead of the Gemnasium API client.

unmarked as a Work In Progress

assigned to @adamcohen and unassigned @fcatteau

marked as a Work In Progress

added 1 commit

• 720e1517 - Apply suggestion to advisory/advisory.go

Compare with previous version

changed the description

added 18 commits

• b24b4d35 - Rename sub-package to depfile
• 6c1ed1ad - Extract client sub-package
• 5cf045f5 - Extract finder sub-package
• fdec38bf - Move to finder.File
• 5b12a413 - Extract Scanner from main func
• a192bc32 - Rename back to scanner pkg
• 03c6374e - Move advisory to scanner/advisory
• 45493175 - Move client to scanner/client
• 0d6a287d - Rename scanner.Source to File
• 71fa6187 - Extract remediate package
• 3b11e19f - Rename VulnerabilityConverter.Source to File
• 45f779ef - Rename scanner.File.FilePath to Path
• a7586263 - Fix finder's ignored directories
• 4a93b5be - Remove DEBUG in Finder
• fc3eb8d1 - Fix & simplify Scanner
• 13962eb6 - Fix test of scanner.parse
• a245ae14 - Fix file path in report
• 8303ff22 - Add advisory.Repo, Advisory

Compare with previous version

changed target branch from master to 14630-pre-refactor

@adamcohen I've applied your suggestion, squashed everything, and rebased on top of 14630-pre-refactor. I'll rebased on top of 14630-pre-refactor as it changes.

added 1 commit

• 68469bdb - Plug in new advisory package, remove client

Compare with previous version

resolved all threads

added 1 commit

• c6f7d9c4 - Fix unit tests

Compare with previous version

added 1 commit

• 8dbd0e42 - Rename to isVersionInRange

Compare with previous version

mentioned in merge request !29 (closed)

assigned to @fcatteau and unassigned @adamcohen

@fcatteau I'll revisit the MR when the tests are working and it's possible to generate a gl-dependency-scanning-report.json

added 1 commit

• fe12b6c7 - Decode package from package slug

Compare with previous version

added 7 commits

• fe12b6c7...77d1b58a - 7 commits from branch 14630-pre-refactor

Compare with previous version

changed target branch from 14630-pre-refactor to 14630-use-gemnasium-db

added 1 commit

• f5787780 - Refactor affectionsFor

Compare with previous version

added 1 commit

• 77d7973d - Add Scanner test, ignore packages w/o advisories

Compare with previous version

@adamcohen Could you have a look at 77d1b58a, f5787780, and 77d7973d?

Again, test for ScanDir is broken but that's because affected ranges are not properly evaluated, and I don't want to tweak the expectations to make it pass (taking the risk of forgetting about it doing so).

assigned to @adamcohen and unassigned @fcatteau

Also, I'd like to move finder/cli.go and advisory/cli.go to scanner/cli.go, and possibly create a new package for what's CLI-related. WDYT?

unmarked as a Work In Progress

mentioned in issue gitlab-org/gitlab#14630 (closed)

approved this merge request

added 1 commit

• 66f698ac - Apply suggestion to advisory/package.go

Compare with previous version

added 1 commit

• 57778c0d - Use CLI flags and ctxt only in scanner.go

Compare with previous version

resolved all threads

approved this merge request

changed the description

mentioned in commit 1c0e8194

merged

mentioned in merge request !25 (merged)

changed the description

added Category:Software Composition Analysis SCA:Dependency Scanning labels