Use native sidecar containers with Kube executor

Summary

When a GitLab runner on Kubernetes runs with sidecar services, it faces an inconsistency. If the runner receives a stop signal from the controller services, the sidecar services can be terminated before the main runner.

Steps to reproduce

  1. Stop a GitLab runner with sidecar services on Kubernetes.
  2. Observe that the termination order of services is random, causing unpredictability in scheduling.

Possible fixes

The recent release of the native sidecar containers feature in Kubernetes version 1.28 provides a potential solution. By utilizing this feature, we can enhance the handling of pod termination. One approach is to move the services to Init containers, ensuring a more orderly shutdown process and resolving the termination order inconsistency.

This issue highlights the need to leverage the new Kube v1.28 sidecar containers feature for better management of pod termination sequencing in GitLab runners on Kubernetes.