Skip to content

docs: Add Note for License Scanning Monorepos

Nicholas Cronquist requested to merge ncronquist/gitlab:license-scanning-monorepos into master

What does this MR do and why?

Describe in detail what your merge request does and why.

  • Documentation change only!
  • Adds a note about how to pass in multiple individual project paths when license scanning a monorepo
  • This argument was not particularly easy to find which is why I thought it was worth documenting. It's not documented in the LicenseFinder README and even though it is available in the license_management CLI help, that's not easy to get to when all you know is that you're using the GitLab CI License Finder job.
    • To view the license_management help, you can run it in docker. Note that the documentation in the license_management cli is not quite correct for the quotes
docker run -it --rm --entrypoint "/bin/bash" registry.gitlab.com/gitlab-org/security-products/analyzers/license-finder:3

# Then run
license_management help report
License Management Help Report Output 
root@5e34efda08d3:~# license_management help report
Usage:
  license_management report

Options:
  -d, [--debug], [--no-debug]                                      # Emit detailed info about what LicenseFinder is doing
  -p, [--prepare], [--no-prepare]                                  # Prepares the project first for license_finder
      [--prepare-no-fail], [--no-prepare-no-fail]                  # Prepares the project first for license_finder but carries on despite any potential failures
  -r, [--recursive], [--no-recursive]                              # Recursively runs License Finder on all sub-projects
  -a, [--aggregate-paths=one two three]                            # Generate a single report for multiple projects. Ex: --aggregate_paths='path/to/project1' 'path/to/project2'
  -q, [--quiet], [--no-quiet]                                      # Silences progress report
      [--columns=one two three]                                    # For text or CSV reports, which columns to print. Pick from: ["name", "version", "authors", "licenses", "license_links", "approved", "summary", "description", "homepage", "install_path", "package_manager", "groups", "texts", "notice"]
      [--format=FORMAT]                                            # Emit detailed info about what LicenseFinder is doing
                                                                   # Default: text
                                                                   # Possible values: text, html, markdown, csv, xml, json, junit
      [--write-headers], [--no-write-headers]                      # Write exported columns as header row (csv).
      [--save=SAVE]                                                # Save report to a file. Default: 'license_report.csv' in project root.
      [--project-path=PROJECT_PATH]                                # Path to the project. Defaults to current working directory.
      [--decisions-file=DECISIONS_FILE]                            # Where decisions are saved. Defaults to doc/dependency_decisions.yml.
      [--log-directory=LOG_DIRECTORY]                              # Where logs are saved. Defaults to ./lf_logs/$PROJECT/prepare_$PACKAGE_MANAGER.log
      [--enabled-package-managers=one two three]                   # List of package managers to be enabled. Defaults to all supported package managers.
                                                                   # Possible values: gomodules, godep, goworkspace, go15vendorexperiment, glide, gvt, govendor, trash, dep, bundler, npm, pip, yarn, bower, maven, gradle, cocoapods, rebar, erlangmk, nuget, carthage, mix, conan, sbt, cargo, dotnet, composer, pipenv, conda, spm
      [--go-full-version=GO_FULL_VERSION]                          # Whether dependency version should include full version. Only meaningful if used with a Go project. Defaults to false.
      [--gradle-include-groups=GRADLE_INCLUDE_GROUPS]              # Whether dependency name should include group id. Only meaningful if used with a Java/gradle project. Defaults to false.
      [--gradle-command=GRADLE_COMMAND]                            # Command to use when fetching gradle packages. Only meaningful if used with a Java/gradle project.
                          Defaults to 'gradlew' / 'gradlew.bat' if the wrapper is present, otherwise to 'gradle'.
      [--maven-include-groups=MAVEN_INCLUDE_GROUPS]                # Whether dependency name should include group id. Only meaningful if used with a Java/maven project. Defaults to false.
      [--maven-options=MAVEN_OPTIONS]                              # Maven options to append to command. Defaults to empty.
      [--npm-options=NPM_OPTIONS]                                  # npm options to append to command. Defaults to empty.
      [--pip-requirements-path=PIP_REQUIREMENTS_PATH]              # Path to python requirements file. Defaults to requirements.txt.
      [--python-version=PYTHON_VERSION]                            # Python version to invoke pip with. Valid versions: 2 or 3. Default: 2
      [--rebar-command=REBAR_COMMAND]                              # Command to use when fetching rebar packages. Only meaningful if used with a Erlang/rebar project. Defaults to 'rebar'.
      [--rebar-deps-dir=REBAR_DEPS_DIR]                            # Path to rebar dependencies directory. Only meaningful if used with a Erlang/rebar project. Defaults to 'deps'.
      [--elixir-command=ELIXIR_COMMAND]                            # Command to use when parsing package metadata for Mix. Only meaningful if used with a Mix project (i.e., Elixir or Erlang). Defaults to 'elixir'.
      [--mix-command=MIX_COMMAND]                                  # Command to use when fetching packages through Mix. Only meaningful if used with a Mix project (i.e., Elixir or Erlang). Defaults to 'mix'.
      [--mix-deps-dir=MIX_DEPS_DIR]                                # Path to Mix dependencies directory. Only meaningful if used with a Mix project (i.e., Elixir or Erlang). Defaults to 'deps'.
      [--sbt-include-groups=SBT_INCLUDE_GROUPS]                    # Whether dependency name should include group id. Only meaningful if used with a Scala/sbt project. Defaults to false.
      [--conda-bash-setup-script=CONDA_BASH_SETUP_SCRIPT]          # Path to conda.sh script. Only meaningful if used with a Conda project. Defaults to '~/miniconda3/etc/profile.d/conda.sh'.
      [--composer-check-require-only=COMPOSER_CHECK_REQUIRE_ONLY]  # Whether to only check for licenses from dependencies on the 'require' section. Only meaningful if used with a Composer project. Defaults to false.

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

I think I may have found a bug in the LicenseFinder CLI where the license found in the aggregate paths aren't exactly the same as those found using the recursive option. Notice the missing jmespath.js and debug licenses in the aggregate image.

I still think it's worth documenting the aggregate-paths option though since you can see even with this very small example, it is twice as fast as the recursive option and as your project dependencies grow, it is even faster. Recursively license scanning a medium sized node monorepo can easily take over an hour and the aggregate option only takes a few minutes.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Nicholas Cronquist

Merge request reports