Adjust rollout strategy for the new DS analyzer and deprecation of build support and Gemnasium analyzer

Problem to solve

Our initial strategy involved upgrading automatically to the new Dependency Scanning analyzer in 18.0 for all the projects we support. This include uses cases where it requires users to modify their CI configuration to generate a new lockfile or graphfile. This was deemed unacceptable for customers. Having only 3 months to adopt a new scanner is too short. Particularly for customers who would need to incur a large effort to apply the necessary changes to use the new DS analyzer (largely attributed to our new way of supporting Java and Python projects).

The purpose of pivoting our rollout strategy is mainly to avoid that disruption for these existing customers.

Yet, to maintain our product roadmap on track and ensure we can continue to improve the Dependency Scanning feature, The groupcomposition analysis group still finds it very important to pursue these two goals:

transition the security analysis engine of Dependency Scanning to the GitLab SBOM Vulnerability Scanner, which already powers our Continuous Vulerability Scanning feature.
reduce the maintenance toll incurred by maintaining the build capability for the Java and Python ecosystems.

To achieve this, we will maintain the removal of the generation of the Dependency Scanning security report artifact from the built-in Dependency Scanning feature. But since we're no longer removing the Gemnasium jobs automatically in 18.0, we need to make it stop outputting (or uploading, TBD) the security report, and only provide an SBOM report like the new DS analyzer. And as of 18.0 all variants of the Gemnasium analyzer will reach end of support.

The feedback received also mentionned another concerns about switching to a non-GA product or not having enough time to validate it once it reaches GA and is applied globally. To address this we have the following possible actions:

We can emphasize the prioritization of this as a roadmap item and ensure we reach GA as soon as possible.
We can nuance the risk by explaining that the GitLab SBOM Vulnerability Scanner engine is in used since 16.10 with Continuous Vulnerability Scanning. Only the adaptation of the CI pipeline workflow is not yet GA. So any concern with regard to the security scanner itself should be reduced.
As a last resort, in GitLab 18.0 users who still don't want the new DS analyzer to run can modifty their CI configuration and set the CI/CD variable DS_EXCLUDED_ANALYZERS to include "dependency-scanning". The exact impact of doing this may vary depending on the chose solution below.

Proposal

After exploring multiple opportunities respecting the constraints above while also trying to upgrade to the new Dependency Scanning analyzer whenever the transition sounds costless (or minimal), I'm suggesting the following options, ordered by an increasing level of risk. All approaches are based on the introduction of new CI/CD variable(s) which control when these analyzer runs, on top of additional criterias. There are also opportunities to fine tune the behavior within each solution, if necessary.

What led to this approach? Why can't we just run both analyzers all the time?

One challenge we're facing is the discrepancy between the files that are supported and scanned by the analyzers and the files that are present in the repository. This is a key element as the CI job rules which determine when a job should run are only checking what's in the repository. This means that some of the files we support for the new DS analyzer won't be present in the repo and thus can't be used to trigger that job. This also prevents the old Gemnasium jobs to know if the new DS analyzer will run or not, and condition their own execution on this. So this leads to two main approaches for managing support overlap:

consider running both analyzers simulteanously
ensure they are mutually exclusives (for a given type of project)

Running both analyzers will theoritically lead to produce the same Security results since they will both use the GitLab SBOM Vulnerability Scanner starting 18.0. But this will actually cause other problems. For instance, having multiple SBOMs reports for the same target can lead to race conditions when processing them in the backend similar to what we're facing with static reachability. This is likely to happen here as the new DS anlyzer provides additional graph data while Gemnasium only provides the list of dependencies. That risk will increase as we futher enrich the SBOM report produced by the new DS analyzer.

We could try addressing this by improving the backend processing logic. However, this sounds risky at first when considering our tight timeline as it requires more development work and it touches the ingestion logic which can have a broader impact. This also involves another product area (Dependency Management) inducing more coordination. Though, at some point we might want to ponder the need to do this, particularly as we extend support to 3rd party SBOMs and want to support some kind of overlap without issues.

On the other end, making these jobs mutually exclusives sounds simpler (at least at first). The complexity will actually depend on the level of granularity we want to have.

1. The pusillanimous

TLDR: only run the new DS analyzer if configured explicitely and run it for all projects (100% opt-in).

Introduce a new CI/CD variables:

DS_ENABLE_NEW_ANALYZER which is set to false by default.

Use this variable to control the jobs execution as follow:

Run gemnasium-maven on the same conditions as today, unless DS_ENABLE_NEW_ANALYZER is set to true.
Run gemnasium-python on the same conditions as today, unless DS_ENABLE_NEW_ANALYZER is set to true.
Run gemnasium (main) on the same conditions as today, unless DS_ENABLE_NEW_ANALYZER is set to true.
Run DS analyzer only if DS_ENABLE_NEW_ANALYZER is set to true.

This approach ensures that all existing users stay with the Gemnasium analyzer by default and the new DS analyzer never runs, unless told explicitely. This is great to avoid disruption for existing users, but not so great for their migration to the new DS analyzer and its adoption by new users. Indeed, new projects will have this behavior too when using the Dependency Scanning CI template to setup the feature. How to best onboard new users can be addressed differently though.

As existing users wish to migrate to the new DS analyzer they can set DS_ENABLE_NEW_ANALYZER to true (at the project, group, or instance level).

Users who want to entirely prevent the use of the new DS analyzer have nothing to do.

This change can be applied in 17.9 to the stable CI template Dependency-Scanning.gitlab-ci.yml, and users who want to try out the breaking changes before 18.0 can do so by setting DS_ENABLE_NEW_ANALYZER to true.

2. The curious

TLDR: Run the new DS analyzer automatically on newly supported projets (vs Gemnasium). Or run it for all projects if configured explicitely.

Introduce a new CI/CD variables:

DS_ENFORCE_NEW_ANALYZER which is set to false by default.

NB: here the variable prefix is "ENFORCE_" rather than "ENABLE_" as the enablement is already automatic for some projects.

Use this variable to control the jobs execution as follow:

Run gemnasium-maven on the same conditions as today, unless DS_ENFORCE_NEW_ANALYZER is set to true.
Run gemnasium-python on the same conditions as today, unless DS_ENFORCE_NEW_ANALYZER is set to true.
Run gemnasium (main) on the same conditions as today, unless DS_ENFORCE_NEW_ANALYZER is set to true.
Run DS analyzer:
- if DS_ENFORCE_NEW_ANALYZER is set to true.
- OR if the repository contains any file that is only supported by the new DS analyzer, but ensure to not scan anything already supported by the gemnasium analyzers using DS_EXCLUDED_PATHS.

This approach ensures that all existing customers stay with the Gemnasium analyzer by default and it enables automatically the new DS analyzer for its newly supported files only. And even if the new DS analyzer is triggered in a monorepository, it won't scan any file related to projects already handled by the Gemnasium analyzers. This is great to avoid disruption for existing users, but not so great for their migration to the new DS analyzer and its adoption by new users. Indeed, new projects will have this behavior too when using the Dependency Scanning CI template to setup the feature. How to best onboard new users can be addressed differently though.

As existng users wish to migrate to the new DS analyzer they can set DS_ENFORCE_NEW_ANALYZER to true (at the project, group, or instance level).

Here are the types of projects that are only supported by the new DS analyzer and for which this analyzer will run automatically in this proposal (when DS_ENFORCE_NEW_ANALYZER is set to false):

C/C++/Fortran/Go/Python/R projects using conda (conda-lock.yml).
Objective-C projects using cocoapods (Podfile.lock).
Rust projects using Cargo (Cargo.lock).
Swift projects using Swift (Package.resolved).

Users who want to entirely prevent the use of the new DS analyzer must set the CI/CD variable DS_EXCLUDED_ANALYZERS to "dependency-scanning".

3. The enthusiast

TLDR: Run the new DS analyzer automatically on projects but Java and Python ones (mainly those requiring a build but some others too). Or run it for all projects if configured explicitely.

Introduce a new CI/CD variables:

DS_ENFORCE_NEW_ANALYZER which is set to false by default.

Use this variable to control the jobs execution as follow:

Run gemnasium-maven on the same conditions as today, unless DS_ENFORCE_NEW_ANALYZER is set to true.
Run gemnasium-python on the same conditions as today, unless DS_ENFORCE_NEW_ANALYZER is set to true.
Run gemnasium (main) only if DS_EXCLUDED_ANALYZERS contains "dependency-scanning".
Run DS analyzer:
- if DS_ENFORCE_NEW_ANALYZER is set to true, and then scan all supported files
- OR if any compatible file that is not already supported by gemnasium-maven and gemnasium-python is found, but ensure to not scan anything already supported by the gemnasium-maven and gemnasium-python analyzers using DS_EXCLUDED_PATHS.

This approach ensures that Java and Python projects (mainly impacted by the lack of "build" capability) from existing customers stay with the Gemnasium analyzer by default. It enables automatically the new DS analyzer for its newly supported files and projects who used to be scanned with the main gemnasium job (roughly those that don't need the build), unless the user explicitely disable the use of the new DS analyzer. And even if the new DS analyzer is triggered it won't scan any file related to projects already handled by the gemnasium-maven and gemnasium-python analyzers.

This is a compromise that still avoids the biggest concern (effort to migrate Java and Python) and improves the adoption of the new Dependency Scanning analyzer, both for existing and new users.

Indeed the technologies for which there is a manual migration step asking to generate a new file are the following:

Java (Gradle, Maven, Sbt)
Python (Pip, Pipenv, Setuptools)
Go* (go.graph). NB: Our Go "build" support is a bit of a special beast, I won't detail why here for the sake of simplicity, but there might be a specific handling needed for it.

So with this solution we would have all these project types using the new DS analyzer out of the box:

C# projects using nuget (packages.lock.json)
C/C++ projects using conan (conan.lock)
C/C++/Fortran/Go/Python/R projects using conda (conda-lock.yml).
JavaScript/TypeScript projects using npm (package-lock.json, npm-shrinkwrap.json)
JavaScript/TypeScript projects using pnpm (pnpm-lock.yaml)
JavaScript/TypeScript projects using yarn (yarn.lock)
Objective-C projects using cocoapods (Podfile.lock).
PHP projects using composer (composer.lock)
Ruby projects using bundler (Gemfile.lock, gems.locked).
Rust projects using Cargo (Cargo.lock).
Swift projects using Swift (Package.resolved).

This solution works great for projects with a basic CI configuraton. However, there is still a risk that customers who have extended the main gemnasium dependency scanning job and/or customized it beyond some extant (e.g. overridding the rules) in their .gitlab-ci,yml will have both analyzers running or some CI configuration adjustments to make in 18.0.

The messaging is also relatively simple: if you're using Python or Java, there is no change without an action from you. If you're using any other technology, you'll be moving to the new DS analyzer automatically in 18.0.

Users who want to entirely prevent the use of the new DS analyzer must set the CI/CD variable DS_EXCLUDED_ANALYZERS to "dependency-scanning". Doing this will re-enable the main gemnasium job too, to ensure these types of projects are scanned.

This change CAN'T be applied in 17.9 to the stable CI template (Dependency-Scanning.gitlab-ci.yml), due to the breaking change for the main gemnasium job. To allow users to try out the breaking changes before 18.0 we could update the stable CI template in 17.9 with option 2, offering an opt-in rollout first where user can test it by setting DS_ENFORCE_NEW_ANALYZER to true. Then we apply the final update for this solution in 18.0.

4. The adventurer

TLDR: Run the new DS analyzer automatically on projects that don't require a build, including some Java and Python particular use cases. Or run it for all projects if configured explicitely.

This is a similar approach to the previous option (3), but even more granular. Indeed, theroritically, we could also consider that for some Python, Java, and Go projects we could also automatically upgrade to the new DS analyzer if they provide the necessary file(s). In such situations the new DS analyzer could be used without requiring the user to do the costly manual step of generating a new file. Here are the project types we could try to automatically migrate to the new DS analyzer:

Python projects using pip having a requirements.txt but only when used as a lockfile.
Python projects using pipenv and having a Pipfile.lock.
Python projects using poetry and having a poetry.lock.
Python projects using UV and having a uv.lock.
Java projects using Maven but having a maven.graph.json.
Java projects using Gradle but having a dependencies.lock.
Java projects using sbt but having a dependencies-compile.dot.
Go projects having a go,graph file.

Some of these uses cases are easy to support, some are more difficult and require complex rules and even file content inspection for the case of requirement.txt. The risk to break CI configuration is increased in this solution due to the likelyhood of advanced customizations for the Java and Python technologies.

Additionally, chosing this option will likely muddy the water as we'll no longer have the clear distinction offered by option 3: Java and Python still using the old Gemnasium analyzer and all the rest are transitioning to the new DS analyzer.

I haven't further explored how to achieve this precisely, spending the time to try defining the rules properly to do so is irelevant if we don't want to pursue this idea.

Onboarding of new customers and migrating existing ones

Independently from the chosen option, as we now have an opt-in approach it will be important to follow-up with actions to incentivize the migration and ensure new users adopt the new Dependency Scanning analyzer. Indeed, by default new users will have the old Gemnasium jobs running when they include the Dependency Scanning CI template (more or less depending on which solution above).

Push to use CI/CD component for the new feature (in the user doc). Though, the lack of maturity of CI/CD components and the possible discrepancy with SEPs (they use CI templates) will add troubles. The situation will hopefully evolve in the coming year.
Add a new CI template dedicated to the new DS with SBOM... (Dependency-Detection.gitlab-ci.yml)? Use this template in the user doc for setting up DS with SBOM, Dependency List, etc...
Advertise targeted users in the UI (not as easy as it sounds apparently)
Advertise targetted customers via different channels (CS, SAs, Support, etc.)
Increase awareness in CI pipeline views with a failing job (allowed to fail, to show a ⚠️ icon) or other opportunities to make it visible for users.
Provide a tool to help users migrate their CI configuration to use the new analyzer
Update the "Configure with a merge request" feature to set the DS_ENFORCE_NEW_ANALYZER var to true
etc

TODOS

What about latest CI template?
Impact on deprecation and breaking changes announcement?

Timeline

For the stable template, no breaking changes must ship before 18.0. Any change that is 100% opt-in can be shipped any time, granted it is correctly documented.

For the latest template, any change, breaking or not, can be shipped anytime. Though, we try to assist as much as possible users who might be disrupted.

timeline
    17.9 : Modify stable DS template with the opt-in changes (option 2)
         : Modify latest DS template with the opt-in changes (option 2)
         : Announce deprecation
         : Release Migration guide
         : DS with SBOM reach Beta (if completed)
    17.10: Release in UI warnings (To Be Confirmed)
    17.11: DS with SBOM reach GA (To Be Confirmed)
    18.0 : Release Gemasium V6. Stop uploading DS report artifact in related CI jobs
         : End of support for Gemnasium analyzers and dependent features

FAQ

Q: Package.resolved is mentionned as scanned only by the new DS analyzer. Is that true? It seems that Gemnasium can also scan this file.
- A: It is true that some files advertised as newly supported in the new DS analyzer are also technically supported by the Gemnasium analyzer. Though, these files are not used in rules:exists to trigger the Gemnasium CI job(s) in the DS CI template. These have been only advertised to be used with the experimental CI/CD components. Using Gemnasium to add support for them was deemed to be the easiest technical solution yet we did not want to officially maintain that support there.

Outcomes

We've decided to go with option 2 and stay with the existing Dependency-Scanning CI template as the way to migrate to the new DS analyzer, by setting the DS_ENFORCE_NEW_ANALYZER CI/CD variable.

We've also decided to maintain the Gemnasium analyzer's ability to generate Dependency Scanning reports, but these will only be available as standard artifacts (artifacts:paths) rather than Report artifacts (artifacts:reports).

We've decided to update the latest template in 17.9 to follow the same opt-in approach. We will possibly envision to set DS_ENFORCE_NEW_ANALYZER to true in 18.0 to switch to the new DS analyzer by default when using this template.

Finally, as the Dependency Scanning CI template will continue to use Gemnasium by default, we will follow up with a plan to encourage existing users to migrate to the new Dependency Scanning analyzer. We will also evaluate the best approaches for onboarding new users directly to the new DS analyzer.

Edited Feb 14, 2025 by Olivier Gonzalez