Make pipeline execution policy variables take precedence

Global variables defined in a pipeline execution policy can currently be overwritten by other settings but should take the highest precedence.

Steps to reproduce

1. Create a group
2. Go to Settings -> General
3. Toggle the Permissions and group features section
4. Enable Run customized CI YAML file as security policy actions
5. Create a project
6. Add a .gitlab-ci.yml file containing a variable:

   variables:
       GLOBAL_VAR: "project setting"

7. Go to Secure -> Policies
8. Select New policy
9. Select Scan execution policy
10. Switch to .yaml mode and paste the following policy:

    type: scan_execution_policy
    name: Compliance
    description: ''
    enabled: true
    rules:
      - type: pipeline
        branch_type: all
    actions:
      - scan: custom
        ci_configuration: |-
          variables:
            GLOBAL_VAR: "policy"
          test policy job:
            stage: test
            script:
              - echo "'$GLOBAL_VAR'"

11. Select Configure with a merge request and merge the MR
12. On the project page, Select Build -> Pipelines
13. Select Run pipeline
14. Add a variable with key: GLOBAL_VAR and value: web trigger
15. Inspect the test policy job logs. The printed variable should say "policy" but it says "web trigger" instead.

Implementation plan

diff --git a/ee/app/services/security/security_orchestration_policies/scan_pipeline_service.rb b/ee/app/services/security/security_orchestration_policies/scan_pipeline_service.rb
index 47667e46cd36..778d967f4001 100644
--- a/ee/app/services/security/security_orchestration_policies/scan_pipeline_service.rb
+++ b/ee/app/services/security/security_orchestration_policies/scan_pipeline_service.rb
@@ -50,6 +50,8 @@ def collect_config_variables(actions, configs)
           variables = action_variables(action)
           jobs = custom_scan?(action) ? Gitlab::Ci::Config.new(config.to_yaml).jobs : config
 
+          variables = variables.merge(config[:variables]) if custom_scan?(action)
+
           jobs&.each_key do |key|
             hash[key] = variables
           end
diff --git a/ee/lib/ee/gitlab/ci/variables/builder/scan_execution_policies.rb b/ee/lib/ee/gitlab/ci/variables/builder/scan_execution_policies.rb
index b67f07851316..89227b47bcf6 100644
--- a/ee/lib/ee/gitlab/ci/variables/builder/scan_execution_policies.rb
+++ b/ee/lib/ee/gitlab/ci/variables/builder/scan_execution_policies.rb
@@ -39,7 +39,10 @@ def variables_for_job(job)
 
             def active_scan_variables(job)
               strong_memoize_with(:active_scan_variables, project) do
-                ci_configs = ::Security::SecurityOrchestrationPolicies::ScanPipelineService.new(ci_context(job))
+                ci_configs = ::Security::SecurityOrchestrationPolicies::ScanPipelineService.new(
+                  ci_context(job),
+                  custom_ci_yaml_allowed: true
+                )
                                                                                            .execute(active_scan_actions)
                 ci_configs[:variables]
               end