renovate-bot: Order of precedence (Vault vs. Variable) breaks infrastructure projects

Background

In the past, Renovate used a token stored in the RENOVATE_GITLAB_TOKEN CI variable. For infrastructure projects, this variable was defined at the gitlab-com/gl-infra group level.

Starting with feat: add support for fetching the Renovate Bot... (!694 - merged), Renovate may now read the token from vault instead, if the right conditions apply.

Problem

The way this is implemented has two problems:

1. The upgrade path is broken for many infrastructure projects. The Vault path is triggered if the VAULT_SECRETS_PATH CI variable is defined. For projects managed through infra-mgmt, this is the case if vault = {enabled = true} is configured. However, the token will only actually exist if common_ci_tasks = {enabled = true} is configured. That means that infrastructure projects which use Vault for anything will break when upgrading common-ci-tasks.

2. Continuing to use the centrally managed token is hard, because the Vault based configuration takes precedence.

   Use case: In Runway we have a merge request pipeline which triggers a "dry-run" deployment in another project. In the past we granted permissions to @glrenovatebot so that these pipelines could run automatically. With PrATs, this is no longer possible. Hence, I'd prefer to continue using the group token.

Suggestion

I suggest that we copy the logic from semantic-release.yml, which uses the following order of precedence:

1. explicit vault input (custom Vault path)
2. SEMANTIC_RELEASE_GITLAB_TOKEN CI variable
3. default Vault path

This would effectively opt-out any infrastructure projects from Vault based authentication, as long as the group access token is available as a group-level CI variable.

If we want to migrate all infrastructure projects to using Vault instead of a CI variable, I think we need to ensure all applicable projects have the common_ci_tasks = {enabled = true} prior to making these sort of changes in common-ci-tasks.