Cancel outdated deployments for jobs with resource groups

What does this MR do and why?

Summary: This ensures that the "Prevent outdated deployment jobs" setting will also apply to deploy jobs with a resource group.

Further details

When enabling the project setting Prevent outdated deployment jobs, deploy jobs in an older pipeline are dropped when they are run. The example flow described below is the normal behavior for most jobs:

• There are 2 running pipelines: pipeline-A and pipeline-B
  • pipeline-A is the older pipeline
  • pipeline-B is the newer pipeline
• The deploy job of pipeline-B runs before the deploy job of pipeline-A (this can happen because the effective execution order of pipeline jobs can vary from run to run)
• The pipeline-B deploy job succeeds
• When pipeline-A deploy job is picked up, it is dropped/failed with a reason of "outdated deployment" (see code)

However, for jobs that have a Resource Group with process_mode=newest_first, the deploy jobs in older pipelines are not dropped/failed.

With this change, we ensure that outdated deploy jobs with a resource group are also dropped/failed.

For more details in this solution, please see: #408981 (comment 1775876163)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

The deploy job from the older pipeline is dropped/failed with a reason of "the deployment job is older than the latest deployment"

How to set up and validate locally

Setup

1. Create a new project or choose an existing project

2. In Project -> Settings -> CI/CD -> General pipelines, check: "Prevent outdated deployment jobs" and "Auto-cancel redundant pipelines"

3. Update the project's CI configuration (.gitlab-ci.yml)

   stages:
     - build
     - deploy
   
   build:
     stage: build
     script:
       - sleep 60
   
   deploy:
     stage: deploy
     environment:
       name: production
       action: start
     resource_group: production
     script:
       - sleep 120

4. Once you've saved the gitlab-ci.yml file, a new pipeline will run. Just let it run, and once it completes, set the process_mode for the 'production' resource group to 'newest_first' (see API guide)

   curl --request PUT \
        --data "process_mode=newest_first" \
        --header "PRIVATE-TOKEN: <your-personal-access-token>" 
        "<your-local-gitlab-url>/api/v4/projects/<your-project-id>/resource_groups/production"

5. Make a new commit in the project's main branch. (This can be any commit, e.g.: add a README file or update the README file).

   • A new pipeline will be kicked off. Let's call this pipeline-A

6. While the first pipeline's build job is running, make another commit in the project's main branch.

   • Another pipeline will be kicked off. Let's call this pipeline-B

7. The jobs will be run in this order:

   • pipeline-A build job
   • pipeline-B build job
   • pipeline-B deploy job
   • pipeline-A deploy job

Test

1. Before: When you are on master, pipeline-A deploy job will run successfully
2. After: When you are on 408981-drop-outdated-deployments-for-jobs-with-resource-groups, pipeline-A deploy job will fail with a reason of "the deployment job is older than the latest deployment" (see screenshots)

Related to #408981 (closed)