Migrate static reachability to semgrep-core

Related issues

https://gitlab.com/gitlab-org/gitlab/-/issues/537156+

Proposal

Update the DS image so that it has semgrep-core. Instead of using gitlab-static-reachability for the SR analysis we can use semgrep-core to generate the same report. The pros of this approach is

• Performance boost due to using semgrep
• Less complex template jobs. We have one job that does DS and SR. This is the best possible solution.

Migration

Once we release this new image of DS it will be used automatically by the latest DS template. This will not cause any issues since we are just adding a binary and a new command. After we experiment with the new version of the latest DS template we can just deploy a newer version of the template and then SaaS beta users will benefit from it.

Implementation plan

• Add semgrep-core in the DS analyzer image

• Add a new command in the DS analyzer image that does the following:

  • generates the semgrep_targets.json
  • generates the semgrep_rules.json
  • runs semgrep-core against those files
  • generate a reachable_packages.json report

• Release DS image

• Modify the latest DS template to:

  • remove gitlab-static-reachability job
  • remove dependency-scanning-with-reachability job
  • The DS job script will:
    • Run DS analyzer to generate SBOM reports
    • Run semgrep-core DS command. This will create the reachable_packages.json file that gitlab-static-reachability job was creating. This and following stages will run if DS_STATIC_REACHABILITY_ENABLED is true .
    • If the reachable_package.json file is present download SCA-to-sarif-matcher and enrich SBOM.

• Update documentation for SR. Be aware that once this issue is released until the new gitlab release we will have 2 ways of doing SR. The old way for self managed that won't be getting the update on the latest template and SaaS that will get everything.