Select the namespace with the CI context when using managed Kubernetes resources

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Release notes

Problem to solve

Currently, a managed resources CI job, using the default template would be similar to:

test_managed_resource:
    image: portainer/kubectl-shell:2.26.1
    variables:
        AGENT_ID: 1087043
        AGENT_CONTEXT: gitlab-org/ci-cd/deploy-stage/demos/infra-platform:demo-dev
    before_script:
        - kubectl config use-context $AGENT_CONTEXT
        - kubectl config set-context --current --namespace=${CI_ENVIRONMENT_SLUG}-${CI_PROJECT_ID}-${AGENT_ID}
    script:
        - kubectl run nginx --image=nginx
    environment:
        name: dev/test_stopped
        kubernetes:
            agent: $AGENT_CONTEXT

We need, two variables, referencing the same agent: AGENT_CONTEXT, AGENT_ID. This is poor UX, especially as the agent ID was meant for troubleshooting primarily, and the user can not figure it out, but needs to look it up somewhere.

Proposal

Idea 4

Note: the comments in thread #521125 (comment 2365313164) explain why Idea 5 is not chosen for the first iteration, but potentially for a later iteration..

Tiger developed a PoC in Draft: Resolve "Select the namespace with the C... (!182830 - closed).

Rails parses the rendered template to find namespaces it manages. If there is exactly one (like in the default template) then Rails would make the CI_KUBE_AGENT_CONTEXT and CI_KUBE_AGENT_NAMESPACE available to the job:

test_managed_resource:
    image: portainer/kubectl-shell:2.26.1
    variables:
        AGENT_CONTEXT: gitlab-org/ci-cd/deploy-stage/demos/infra-platform:demo-dev
    before_script:
        - kubectl config use-context "${CI_KUBE_AGENT_CONTEXT}"
        - kubectl config set-context --current --namespace="${CI_KUBE_AGENT_NAMESPACE}"
    script:
        - kubectl run nginx --image=nginx
    environment:
        name: dev/test_stopped
        kubernetes:
            agent: $AGENT_CONTEXT

Benefits:

  • Less plumbing required (namespace) by the user compared to the other ideas above.

Caveats:

  • it still requires plumbing by the user for no or multiple namespaces. I'd consider that scenario an edge case.

Next iteration

Idea 5

Rails parses the rendered template to find namespaces it manages. If there is exactly one (like in the default template) then the injected kubeconfig will already have the namespace set in the context for the selected agent (in addition, we could set that as the current context). This would provide a clean YAML:

test_managed_resource:
    image: portainer/kubectl-shell:2.26.1
    script:
        - kubectl run nginx --image=nginx
    environment:
        name: dev/test_stopped
        kubernetes:
            agent: gitlab-org/ci-cd/deploy-stage/demos/infra-platform:demo-dev

We may still expose the CI_KUBE_AGENT_CONTEXT and CI_KUBE_AGENT_NAMESPACE variables to the job.

Benefits:

  • Like Idea 4, but even less plumbing.

Caveats:

  • it still requires plumbing by the user for no or multiple namespaces. I'd consider that scenario an edge case.
Expand for alternative *rejected* ideas

Idea 1

When the agent is configured for managed resources, auto-select the namespace from the default template in the context.

This would allow the following to work:

test_managed_resource:
    image: portainer/kubectl-shell:2.26.1
    variables:
        AGENT_CONTEXT: gitlab-org/ci-cd/deploy-stage/demos/infra-platform:demo-dev
    before_script:
        - kubectl config use-context $AGENT_CONTEXT
    script:
        - kubectl run nginx --image=nginx
    environment:
        name: dev/test_stopped
        kubernetes:
            agent: $AGENT_CONTEXT

Caveats:

  • we need to figure out the namespace from the default template, even when it was overwritten
  • what if the default template does not contain a namespace definition
  • the job still needs an extra command when uses a non-default template with a different namespace pattern
  • can we handle situations when the same pipeline has multiple environments, some with, other without managed resources? thus some would not need the namespace selection

Idea 2

Introduce new agent context, like <project/slug>:<my-agent>:managed-resource

This would allow the following to work:

test_managed_resource:
    image: portainer/kubectl-shell:2.26.1
    variables:
        AGENT_CONTEXT: gitlab-org/ci-cd/deploy-stage/demos/infra-platform:demo-dev:managed-resource
    before_script:
        - kubectl config use-context $AGENT_CONTEXT
    script:
        - kubectl run nginx --image=nginx
    environment:
        name: dev/test_stopped
        kubernetes:
            agent: $AGENT_CONTEXT

Caveats:

  • we need to figure out the namespace from the default template, even when it was overwritten
  • what if the default template does not contain a namespace definition
  • the job still needs an extra command when uses a non-default template with a different namespace pattern
  • can we handle situations when the same pipeline has multiple environments, some with, other without managed resources? thus some would not need the namespace selection

Idea 3

Every time when environment.kubernetes.agent is set, inject a predefined CI variable, CI_KUBE_AGENT_ID, that the user can use in their scripts.

This would allow the following to work:

test_managed_resource:
    image: portainer/kubectl-shell:2.26.1
    variables:
        AGENT_CONTEXT: gitlab-org/ci-cd/deploy-stage/demos/infra-platform:demo-dev
    before_script:
        - kubectl config use-context $AGENT_CONTEXT
        - kubectl config set-context --current --namespace=${CI_ENVIRONMENT_SLUG}-${CI_PROJECT_ID}-${CI_KUBE_AGENT_ID}
    script:
        - kubectl run nginx --image=nginx
    environment:
        name: dev/test_stopped
        kubernetes:
            agent: $AGENT_CONTEXT

Caveats:

  • the user needs to set the namespace explicitly

Intended users

The user setting up pipelines:

Feature Usage Metrics

No new metrics are required

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

Edited by 🤖 GitLab Bot 🤖