Allow GitLab Runner to process named PodSpec(s) set in a .gitlab.ci yml pipeline file
## Overview
Enabling support for named GitLab Runner PodSpec(s) to configure the Runner worker will unlock value for users of GitLab for [scientific workflows](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3114#note_1302779385).
For example, data scientists, machine learning researchers, or engineers will require GPU-enabled computing on public cloud Kubernetes infrastructure. To use GPU-enabled compute means creating different podspecs for each class of GPU-enabled compute offered. Today, the podspec can only be set in the runner config.toml file.
So first - this means the administrator of the self-managed Runner has to configure the podspec for each compute type. Secondly, every job on the specified Runner will request GPU-enabled resources regardless of whether the job requires a GPU. As a workaround, customers can configure multiple runners, each with its own pod spec. That approach is not efficient or flexible.
## Value proposition (customer verbatims)
- "Here is one specific example: say we added a [new type of node](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#clusters-containing-different-types-of-gpus) to a Kubernetes cluster, and we wanted to use a node selector to run a pod on that specific node. It would be easier for the CI pipeline developers to make a code change (`.gitlab-ci.yml`) than to have to contact a system administrator to modify the Runner's configuration (`config.toml`) to add a new named PodSpec.
- The workaround of `multiple names pod specs` is not ideal as "there are common situations that have a number of permutations. Take, for example, the GPU specification for GKE Autopilot workloads as shown in the [Google documentation](https://cloud.google.com/kubernetes-engine/docs/how-to/autopilot-gpus#request-gpus). If we wanted to cover all GPU type/quantity options, that would be eight unique pod specs. Not an intractable number, but slightly cumbersome."
## Proposal
Allow users to configure a podspec in the `.gitlab-ci.yml` pipeline.
## Example
### config.toml
```toml
...
[[runners]]
[runners.kubernetes]
[[runners.kubernetes.pod_spec]]
name = "spec-a100-3"
patch = '''
nodeSelector:
cloud.google.com/gke-accelerator: "nvidia-tesla-a100"
containers:
- name: "build"
resources:
limits:
nvidia.com/gpu: 2
'''
patch_type = "strategic"
...
```
### gitlab-ci
```yaml
pod_specs:
- name: spec-a100-1
patch_type: "strategic"
patch: |
hostname: "my_host"
- name: spec-a100-2
patch_type: "json"
patch: |
[{"op": "replace", "path": "/terminationGracePeriodSeconds", "value": 60}]
job01:
pod_specs:
- spec-a100-2
script:
- ./cmd01
- ./cmd02
job02:
pod_specs:
- spec-a100-1
- spec-a100-3
script:
- ./cmd01
- ./cmd02
job03:
script:
- ./cmd01
- ./cmd02
services:
- name: my-postgres:11.7
alias: db-postgres
entrypoint: ["/usr/local/bin/db-postgres"]
command: ["start"]
- pod_specs:
- spec-a100-1
- spec-a100-3
```
issue