Draft: Test version parsing against export v1 of licenses
What does this MR do?
Test version parsing against export v1 of license data.
This is built on top of Draft: Test parsing and comparison against expo... (!85).
To speed things up and to produce a readable output, it only count what's correct, and reports the percentage of correct results if not 100%.
See Integration tests for semver_dialects using lic... (gitlab-org/gitlab#462854 - closed)
Usage
First you need to copy license data from the GCP bucket locally.
You only need the v1
directory, and can skip v2
.
You can follow the instructions for offline environments.
The environment variable LICENSES_BUCKET_DIR
must contain the path of the local copy.
PKG_METADATA_BUCKET="prod-export-license-bucket-1a6c642fc4de57d4"
LICENSES_BUCKET_DIR="$HOME/license-db/$PKG_METADATA_BUCKET/"
gsutil -o "GSUtil:parallel_process_count=1" -m rsync -r -d -y "^v2\/" "gs://$PKG_METADATA_BUCKET" "$LICENSES_BUCKET_DIR"
The following line runs the specs using all the CSV files found LICENSES_BUCKET_DIR
and all supported package types.
bundle exec rspec spec/license_scanning_spec.rb
You can limit the checks to N lines/JSON objects per package type using LINES_PER_PKG_TYPE
.
LINES_PER_PKG_TYPE=10000 bundle exec rspec spec/license_scanning_spec.rb
You can use LICENSES_PKG_TYPES
to only check specific package types.
These types correspond to sub-directories of v1
, not to the ones used by the gem and by gemnasium-db.
LICENSES_PKG_TYPES="conan rubygem" bundle exec rspec spec/license_scanning_spec.rb -f p