Kubernetes namespace dropdown list is empty on Edit environment page

Summary

When using impersonation for the Kubernetes agent, when editing the environment, Kubernetes namespace shows error Forbidden to access the cluster agent from this environment. and the dropdown list is empty even when RBAC role is in place for a given namespace.

User is able to manually type the namespace and save changes.

Steps to reproduce

1. Create a GitLab Kubernetes agent configuration project and use impersonation. Install the agent. My agent config is as below:

ci_access:
  groups:
    - id: tests
user_access:
  access_as:
    user: {}
  groups:
    - id: tests
  projects:
    - id: tests/deploy-with-kas-agent

Note: tests group id is 2.

2. Create RBAC resources. In my example project, I've used:

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: gitlab-agent-cluster-role
rules:
- apiGroups: [""]
  resources: ["*"]
  verbs: ["get", "watch", "list", "create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: gitlab-agent-dotcom-runners-role-binding
  namespace: dotcom-runners 
roleRef:
  name: gitlab-agent-cluster-role
  kind: ClusterRole
  apiGroup: rbac.authorization.k8s.io
subjects:
  - name: gitlab:group_role:2:developer
    kind: Group
  - name: gitlab:group_role:2:owner
    kind: Group
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: gitlab-agent-nginx-role-binding
  namespace: nginx
roleRef:
  name: gitlab-agent-cluster-role
  kind: ClusterRole
  apiGroup: rbac.authorization.k8s.io
subjects:
  - name: gitlab:group_role:2:developer
    kind: Group
  - name: gitlab:group_role:2:owner
    kind: Group
---

3. Create a new project in tests group. Deploy to the nginx namespace using the agent. Sample CI/CD script is:

deploy-nginx:
  stage: deploy
  image: 
    name: 'dtzar/helm-kubectl:latest'
    entrypoint: ['']
  environment:
    name: dev
    url: https://nginx.supermunn.net
  script:
    - kubectl config use-context tests/kas-agent:elif-test
    - helm repo add bitnami https://charts.bitnami.com/bitnami
    - helm repo update
    - helm upgrade --install nginx bitnami/nginx --namespace nginx --create-namespace --wait

4. When deployment is complete, go to Operate > Environments. Select the environment from Step 3 (in my case, it is dev). Then click Edit environment.
5. Open the Kubernetes namespace dropdown. It shows empty stating No results found.
6. You can manually type nginx and click the Or select namespace nginx.

What is the current bug behavior?

The namespaces that can be accessed are not displayed in the dropdown list. This is misleading for the user, making the user think there are no namespaces that can be accessed.

What is the expected correct behavior?

The namespaces that can be accessed should be displayed in the dropdown list.

Relevant logs and/or screenshots

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info

System information
System:		Debian 12
Proxy:		no
Current User:	git
Using RVM:	no
Ruby Version:	3.1.5p253
Gem Version:	3.5.11
Bundler Version:2.5.11
Rake Version:	13.0.6
Redis Version:	7.0.15
Sidekiq Version:7.1.6
Go Version:	unknown

GitLab information
Version:	17.3.7-ee
Revision:	d8f6f167415
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	14.11
URL:		https://gitlab-single.supermunn.net
HTTP Clone URL:	https://gitlab-single.supermunn.net/some-group/some-project.git
SSH Clone URL:	git@gitlab-single.supermunn.net:some-group/some-project.git
Elasticsearch:	no
Geo:		no
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers: 

GitLab Shell
Version:	14.38.0
Repository storages:
- default: 	unix:/var/opt/gitlab/gitaly/gitaly.socket
GitLab Shell path:		/opt/gitlab/embedded/service/gitlab-shell

Gitaly
- default Address: 	unix:/var/opt/gitlab/gitaly/gitaly.socket
- default Version: 	17.3.7
- default Git Version: 	2.45.2

Results of GitLab application Check

Expand for output related to the GitLab application check

Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 14.38.0 ? ... OK (14.38.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes (cluster/worker) ... 1/1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Database config exists? ... yes
Tables are truncated? ... skipped
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Cable config exists? ... yes
Resque config exists? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Systemd unit files or init script exist? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Systemd unit files or init script up-to-date? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Projects have namespace: ...
2/1 ... yes
2/2 ... yes
1/3 ... yes
2/4 ... yes
2/5 ... yes
2/6 ... yes
2/7 ... yes
2/8 ... yes
2/9 ... yes
2/10 ... yes
2/11 ... yes
2/12 ... yes
Redis version >= 6.2.14? ... yes
Ruby version >= 3.0.6 ? ... yes (3.1.5)
Git user has default SSH configuration? ... yes
Active users: ... 2
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Elasticsearch version 7.x-8.x or OpenSearch version 1.x ... skipped (Advanced Search is disabled)
All migrations must be finished before doing a major upgrade ... skipped (Advanced Search is disabled)
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished

Possible fixes

• Edit environment. Then open the Kubernetes namespace dropdown list
• Type in the namespace and click the Or select namespace: <NAMESPACE>

• Click Save. You will see the pods in the namespace:

added devopsdeploy groupenvironments typebug labels

added sectioncd label

Large GitLab Ultimate customer reported this via Support ticket - internal only

added customer label

All customer typebug issues need to have a proper severity label set. Please add a severity label, remove the automation:customer-bug-missing-labels label, and then reply to this comment briefly explaining your reasoning for providing this severity.

If you are not the DRI for this area and would like help determining the best severity, please @ the appropriate person for assistance.

This message was generated automatically. You're welcome to improve it.

added automation:customer-bug-missing-labels label

added frontend label

mentioned in issue gitlab-org/quality/triage-reports#21441 (closed)

mentioned in issue gitlab-org/quality/triage-reports#21516 (closed)

mentioned in issue gitlab-org/quality/triage-reports#21629 (closed)

mentioned in issue gitlab-org/quality/triage-reports#21701 (closed)

added UX severity2 labels

removed automation:customer-bug-missing-labels label

added priority2 label

mentioned in issue gitlab-org/quality/triage-reports#21816 (closed)

@e_munn This is the expected behavior as the user does not have RBAC configured to list namespaces. They can still see inside a namespace. Allowing the listing of namespaces requires a ClusterRoleBinding

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: namespace-lister
rules:
- apiGroups: [""]
  resources: ["namespaces"]
  verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: namespace-lister-binding
subjects:
  - name: gitlab:group_role:2:developer
    kind: Group
  - name: gitlab:group_role:2:owner
    kind: Group
roleRef:
  kind: ClusterRole
  name: namespace-lister
  apiGroup: rbac.authorization.k8s.io

I'm closing the issue as the functionality (likely) works as expected. Please, reopen it if needed.

closed