Ability to track/link dependencies.lock file to source material for SBOM Dependency Scanning

Problem to solve

For the SBOM based dependency scanner, the dependency list identifies a location for where the dependency is called, as well as a link to a dependencies.lock file. As noted in the attached screen shot, in some cases the dependency tree is not available, so all we have to track the source of the dependency is the aforementioned lock file.

Unfortunately, for Java/Gradle projects, this dependency.lock file is a transient artifact within the pipeline and not resident within the repository. Therefore, the link within the dependency list is effectively broken, leaving the user with no way to trace the source of the dependency.

This becomes more problematic with projects where multiple gradle build files are present in subdirectories. In more of a mono-repo type architecture, it can be nearly impossible to find where the referenced dependency is pulled in. This also means that any vulnerabilities associated with that vulnerability are difficult to address - if we can't find where that dependency is called, it's difficult to change/upgrade that dependency.

Intended users

• Amy (Application Security Engineer)
• Alex (Security Operations Engineer)
• Cameron (Compliance Manager)-->

User experience goal

Ideally, every dependency within the dependency list can be tracked to where that dependency was injected/incorporated. This means not only having the full dependency tree, but also providing a link to specifically what build.gradle file identifies that dependency.

In some cases, the customer may have resolve rules that change the dependency at build time. The user should still be able to map that dependency back to the proper sub-project in which that dependency is utilized.

Proposal

Two items should be addressed:

1. The dependency tree should always exist for a particular dependency. If a dependency tree is unavailable, the job output should indicate why the tree was unavailable or at least acknowledge it wasn't deterministic.
2. Including a link to a dependency.lock file that doesn't exist isn't very helpful. Because with SBOM based scanning, the dependency lock may include materials from multiple build.gradle files, the link should bring the user back to the relevant build.gradle file.

Further details

Permissions and Security

Documentation

Availability & Testing

Available Tier

• Ultimate

Feature Usage Metrics

What does success look like, and how can we measure that?

Every dependency listed in the dependency list should be traceable back to the source of where that dependency is brought in and what portion of code that utilizes that dependency.

Is this a cross-stage feature?

What is the competitive advantage or differentiation for this feature?

This will improve our new SBOM based dependency scanning developed by groupcomposition analysis

Links / references

Technical proposals

Add a file system check in sbom occurrence ingestion. In input_file_path if the report_source.input_file_path is not present in the repo, then select a corresponding build file based on heuristic. Check if build file also present.

1. Repository look-up

Sbom ingestion will have to be updated to look-up the presence of the files in the project's repository. 2 methods are possible for lookups:

1. Use project.repository.tree(pipeline.sha) in OccurrenceMap allowing it to issue a lookup like: tree.blobs.find{ |obj| obj.path == report_source.input_file_path} (example).
   1. Pro: Single call to repository.
   2. Con: Too many projects files will blow up memory.
2. Use project.repository.search_files_by_name(repor_source.input_file_path, pipeline.sha) to issue a look up for each report_source.input_file_path (example).
   1. Pro: Keeps ingestion memory virtually unchanged.
   2. Con: A monorepo with sub-projects will issue a call for each supplied input_file_path.

At first glance 2 looks less desirable but if it takes a regex then a single query is all that is needed for both.

• Needs investigation

Additional: Build to lock file mapping

To look up a lock file's alternative a mapping is necessary from lock files to their corresponding build files. For example

package manager	lock file	build file
nebula	dependencies.lock	build.gradle
go	go.graph	go.mod

• Table above needs completion.

3. User selects whether to use build file as component source

This option allows user to select whether they want the build file or graph file as input. This is a simple and flexible approach which is easy to maintain and doesn't add any scalability concerns (unlike 1).

Proposed name for parameter: use_build_file_as_source

Allow user to supply parameter (use_build_file_as_source) to CI/CD component or the analyzer (USE_BUILD_FILE_AS_SOURCE) specifying whether to use build file.

When this parameter is supplied, the analyzer will use the build file (e.g. requirements.in or build.gradle) even if the lock file is present.

Conclusion

The user-supplied parameter proposal is the MVC path here because it ensures the Dependency List components are linkable without affecting monolith infrastructure and scalability.

Follow-up

A workaround is necessary for source files that the analyzer can't know about explicitly (requirements.dev.in vs requirements.in vs pyproject.toml or setup.py).

A dependency scanning post-processing step in the user pipeline is the best option here because the user can customize the sbom for their project specifically and doesn't require changes on the rails or analyzer side.

Implementation plan

Option 1: Analyzer overrides the input_file in the sbom

1. Update analyzer to accept DS_EXPERIMENTAL_USE_SOURCE_FILE_FOR_DEPENDENCY_LOCATION. When this parameter is set, the analyzer will use the project.RequirementsFile as scannable. This file will be use for the gitlab:dependency_scanning:input_file:path property in the sbom (This should apply to any supported lockfile found in the project.)

✅ added in: https://gitlab.com/gitlab-org/security-products/analyzers/dependency-scanning/-/releases/v0.17.0

Note: this variable is experimental and subject to removal without notice.

✅ Option 2: Give user a workaround to set the sbom input file as needed

1. Create update script for sbom (update.rb).
   1. Search for all gl-sbom*.cdx.json.
   2. For each sbom.
      1. If gitlab:dependency_scanning:package_manager:name is supported.
      2. Update gitlab:dependency_scanning:input_file:path with the desired source file.
2. Show example ci configuration to modify sbom after analyzer has run (.gitlab-ci.yml).
   1. Add an sbom update job after dependency scanning runs.
   2. Invoke above script.
   3. Update dependency scanning job to export sboms as artifacts but not cyclonedx reports (to avoid duplicates).
   4. Export processed sboms as cyclonedx reports.