Fail only the job when a group secret source is not accessible

Erick Bajaorequested to merge
eb-fix-633509 into master

What does this MR do and why?

Problem

A pipeline fails to be created when a Secrets Manager secret uses a group/... source that is not the project's group or one of its parent groups. Users get a YAML error and no jobs run.

This happens most often in merge requests from forks. The fork lives in another namespace, so the upstream group is outside its hierarchy. The whole pipeline is blocked, even when the job that needs the secret is manual or would never run. HashiCorp Vault fails only the job in the same situation.

Solution

Behind the secrets_manager_defer_group_source_check flag (gitlab_com_derisk, default off):

  1. Config check: a group source outside the hierarchy no longer causes a config error. The pipeline is created and other jobs run.
  2. Runner payload: EE::Ci::BuildRunnerPresenter already looks up the group in the project hierarchy. When the group is not found, it now raises SecretsManagement::SourceNotAccessibleError instead of sending an empty secret config. EE::Ci::RegisterJobService maps the error to a new failure reason, secrets_manager_source_not_accessible, and drops the job. The runner never gets the job, so the script never runs and no secret is exposed. The job page shows a clear message with a link to the docs.

With the flag off, behavior is the same as today.

Other notes:

  • The job is dropped after runner assignment, not before. So it goes pending -> running -> failed, and a job running webhook is sent. This is a tradeoff to avoid an extra query. A check in pre_assign_runner_checks (like secrets_manager_access_denied) would drop the job while it is still pending, but it would add one more self_and_ancestors lookup on every pickup of a job with group/ secrets. The presenter already does this lookup, so raising there costs nothing extra.

  • There is no pipeline warning for the skipped check. Pipeline warnings are kept in memory only, so for a pipeline triggered by a push nobody would see it. The job failure is the signal users see.

  • The new failure reason is not auto-retried. Retrying gives the same result.

  • Project sources (the default, or source: project) are not affected. They always read from the project that runs the pipeline, so they never block pipeline creation.

Database

No new queries and no migrations. The runner payload reuses its existing group lookup.

Rollback

Turn off the flag. Jobs created while the flag was on and not yet picked up go to the runner with an empty secret config and fail with the older runner error. Only that job fails, and no secret is exposed.

References

Screenshots or screen recordings

No new UI. The job failure message uses the existing component.

How to set up and validate locally

Prerequisite: Secrets Manager is set up on GDK, with a runner, and a group group-a that has a group secret MY_SECRET. See doc/development/secrets_manager/local_setup.md.

  1. Create a project in a different group, for example group-b/my-project, with this .gitlab-ci.yml:

    needs-secret:
  secrets:
    MY_SECRET:
      gitlab_secrets_manager:
        name: MY_SECRET
        source: group/group-a
  script: echo "should not run"

other-job:
  script: echo "runs"

  2. Run a pipeline. It fails to be created with source group with path 'group-a' not found.

  3. In the Rails console, enable the flag:

    Feature.enable(:secrets_manager_defer_group_source_check)

  4. Run a pipeline again. It is created with both jobs.

  5. Check that other-job passes and needs-secret fails with secret source group is not accessible from this project.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Erick Bajao

Merge request reports