### Problem We introduced custom rulesets for SAST, Secret Detection, and IaC Scanning in %14.8. This feature was intended to help organizations express their preferences about the pre-packaged rules included in the various scanning tools. Relatedly, this would help avoid forcing users into a "one size fits all" ruleset when local context (like compensating controls) or risk judgements (like different views of severity or applicability of a class of vulnerability) would provide a better experience for the given org. However, the only way to use this feature is through a configuration file committed in the project that's being scanned. This limits the feature in at least two ways: 1. It's too hard to orchestrate this configuration across many repositories. Many organizations have hundreds or thousands of repositories and don't want to make the same customizations everywhere. 2. It's difficult to control this configuration in a central way to ensure separation of duties. Without additional configuration, anyone who has write access to the repository can change the ruleset used for scanning. An existing feature proposal, https://gitlab.com/gitlab-org/gitlab/-/issues/257928+, has seen significant traction. It is clear that we need to deliver an initial capability in this area. ### Proposal Introduce a new CI/CD variable corresponding to each type of supported configuration file: - `SAST_RULESET_GIT_REFERENCE` to reference an external project's `.gitlab/sast-ruleset.toml`, which is [used in SAST](https://docs.gitlab.com/ee/user/application_security/sast/customize_rulesets.html#create-the-configuration-file) and [IaC Scanning](https://docs.gitlab.com/ee/user/application_security/iac_scanning/#customize-rulesets) - `SECRET_DETECTION_RULESET_GIT_REFERENCE` to reference an external project's `.gitlab/secret-detection-ruleset.toml`, which is [used in Secret Detection](https://docs.gitlab.com/ee/user/application_security/secret_detection/#custom-rulesets) This variable would fetch the file and overwrite any local configuration. This is to achieve separation of duties, so that a central security team using Scan Execution Policies can have confidence in the ruleset being used for the scan job. Alternatives will be discussed in comments. ### Not in scope yet We should eventually, but won't as part of this iteration: - Introduce a UI for rule management. This would unduly delay initial delivery and require significant UX design. - Introduce a dedicated instance-level setting for this as a default setting. As an alternative, it will be possible to use this variable in both Compliance Frameworks and Scan Execution Policies, which each cause scans to run across sets of projects. - Automatically detect custom configuration on policy project if associated - Support custom paths for ruleset references (rather than relying on referenced project's default ruleset path ### Implementation plan * [x] ~backend add `ruleset.LoadRemote()` function for loading remote configurations | https://gitlab.com/gitlab-org/security-products/analyzers/ruleset/-/merge_requests/15 * [x] ~backend update [`ruleset.Load()`](https://gitlab.com/gitlab-org/security-products/analyzers/ruleset/-/blob/d019b7464c5301fbd3ef13a55231e904e6a719b3/ruleset.go#L218) to prioritize `SAST_RULESET_GIT_REFERENCE` or `SECRET_DETECTION_RULESET_GIT_REFERENCE` (if present) over local path | https://gitlab.com/gitlab-org/security-products/analyzers/ruleset/-/merge_requests/15 * [x] Bump ruleset dependency for all analyzers * [x] ~documentation Update custom ruleset docs with new variable configuration | https://gitlab.com/gitlab-org/gitlab/-/merge_requests/120920 ### Related issues https://gitlab.com/gitlab-org/gitlab/-/issues/257928+ #### Usecase - Given a group `GroupX`, I want to use a common custom ruleset for `GroupX/ProjectA` and `GroupX/ProjectB`. - I create a `ProjectC` to store my custom ruleset configuration and set a group-level CI variable to reference my custom configuration. - Alternatively, I set the variable within my Security Execution Policy to reference my custom configuration. ``` GroupX SAST_RULESET_FILE_URL=gitlab.example.com/groupx/projecta/-/raw/main/.gitlab/sast-ruleset.toml ProjectA (policies project) .gitlab/ ├── security-policies/ │ └── policy.yml └── sast-ruleset.toml ProjectB ./ ├── app.rb └── README.md ProjectC ./ ├── app.rb └── README.md ``` #### Simple TOML example ```toml # ProjectA .gitlab/sast-ruleset.toml [kics] [[kics.ruleset]] disable = true [kics.ruleset.identifier] type = "kics_id" value = "8212e2d7-e683-49bc-bf78-d6799075c5a7" [[kics.ruleset]] disable = true [kics.ruleset.identifier] type = "kics_id" value = "b03a748a-542d-44f4-bb86-9199ab4fd2d5" ``` See [IaC Scanning: Customize rulesets](https://docs.gitlab.com/ee/user/application_security/iac_scanning/#customize-rulesets) for customization details/TOML format. #### Advanced TOML example ```toml # ProjectA .gitlab/sast-ruleset.toml [semgrep] # Passthrough1 [[semgrep.passthrough]] type = "git" value = "https://gitlab.example.com/user/passthrough1.git" ref = "refs/heads/main" # Passthrough2 [[semgrep.passthrough]] type = "file" value = "passthrough2.yml" # Passthrough3 [[semgrep.passthrough]] type = "url" value = "https://semgrep.dev/c/p/gosec" target = "passthrough3.yml" # Passthrough4 [[semgrep.passthrough]] type = "raw" target = "passthrough4.yml" value = """ rules: - id: "insecure" patterns: - pattern: "func insecure() {...}" message: | Insecure function 'insecure' detected metadata: cwe: "..." severity: "ERROR" languages: - "go" """ ``` See [SAST: Customize rulesets](https://docs.gitlab.com/ee/user/application_security/sast/customize_rulesets.html) for customization details/TOML format. <!-- start-discoto-summary --> ## Auto-Summary :robot: <details> <summary>Discoto Usage</summary> --- > **Points** > > Discussion points are declared by headings, list items, and single > lines that start with the text (case-insensitive) `point:`. For > example, the following are all valid points: > > * `#### POINT: This is a point` > * `* point: This is a point` > * `+ Point: This is a point` > * `- pOINT: This is a point` > * `point: This is a **point**` > > Note that any markdown used in the point text will also be propagated > into the topic summaries. > > **Topics** > > Topics can be stand-alone and contained within an issuable (epic, > issue, MR), or can be inline. > > Inline topics are defined by creating a new thread (discussion) > where the first line of the first comment is a heading that starts > with (case-insensitive) `topic:`. For example, the following are all > valid topics: > > * `# Topic: Inline discussion topic 1` > * `## TOPIC: **{+A Green, bolded topic+}**` > * `### tOpIc: Another topic` > > **Quick Actions** > > | Action | Description | > |-------------------------------|---------------------------------------------------------| > | `/discuss sub-topic TITLE` | Create an issue for a sub-topic. Does not work in epics | > | `/discuss link ISSUABLE-LINK` | Link an issuable as a child of this discussion | > --- </details> Last updated by [this job](https://gitlab.com/gitlab-org/secure/pocs/discoto-runner/-/jobs/4150306663) * **TOPIC** URL, git ref, more generic format? https://gitlab.com/gitlab-org/gitlab/-/issues/393452#note_1295948132 * **TOPIC** Timeline/scope https://gitlab.com/gitlab-org/gitlab/-/issues/393452#note_1295948706 * **TOPIC** Alternatives https://gitlab.com/gitlab-org/gitlab/-/issues/393452#note_1295949897 * **TOPIC** Threat model/potential for evasion of control https://gitlab.com/gitlab-org/gitlab/-/issues/393452#note_1295954778 * **TOPIC** Limiting support for relative passthroughs https://gitlab.com/gitlab-org/gitlab/-/issues/393452#note_1297675190 <!-- end-discoto-summary --> <!-- start-discoto-topic-settings --><details> <summary>Discoto Settings</summary> <br/> ```yaml --- summary: max_items: -1 sort_by: created sort_direction: ascending ``` See the [settings schema](https://gitlab.com/gitlab-org/secure/pocs/discussion-automation#settings-schema) for details. </details> <!-- end-discoto-topic-settings -->