Users should not be able to disable Secret Detection/Container Security/etc. Scans

Summary

As reported in this issue users are able to disable Secret Detection scan execution policies by setting SECRET_DETECTION_DISABLED: true. This should not be allowed as scan execution policies should always run. We should be overriding this variable when we create the secret detection job.

Note: We should check the behavior for the other scanners as well as part of this issue.

Steps to reproduce

Example Project

What is the current bug behavior?

What is the expected correct behavior?

Relevant logs and/or screenshots

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of: sudo gitlab-rake gitlab:check SANITIZE=true)

(For installations from source run and paste the output of: sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)

(we will only investigate if the tests are passing)

Possible fixes

From !105492 (comment 1197589627):

  1. Make the change in Gitlab::Ci::Variables::Builder because it's the SSoT for variables and allows us to memoize common variables among jobs.
  2. Memoize active_scan_policies? at project level so we won't execute it for hundreds of jobs in the pipeline.
  3. Concat variables from ScanPipelineService::SCAN_VARIABLES at the end of Variables::Builder#scoped_variables collection. No need to export to to_hash and re-initialize a collection because that's very expensive. Variables concatenated at the end take precedence when exported with to_hash.
Edited by Dominic Bauer