Missing SSH capability for GKE nodes
Problem Description
We need the ability to ssh into GKE nodes for troubleshooting. At the moment, this capability appears to be limited by 2 things:
- #25851 (closed) - let's assume we fix this issue for the context of this issue - because we can gain ssh access by using our admin accounts for these GCP Projects.
- I suspect a firewall rule
Again, assuming item 1 is fixed, we can freely ssh into the gitaly nodes. So we know ssh access is capable, but for some reason, we end up with a session timeout when attempting to ssh into GKE nodes. Firewall rules from an active Cell:
Not many rules! And the rule k8s-fw-a91c89394b50948d0aaddbbd0ab6eeb1 is specific to the Ingress to enable ssh to GitLab Shell. We have no rule for port 22 to our GKE nodes.
Rule c01j2t2v563b55mswz-ssh targets our Gitaly nodes specifically.
I suspect we need a dedicated rule for GKE.
Action Items
-
Add firewall rules to allow IAP access to all the nodes. 👉 gitlab-org/gitlab-environment-toolkit!1567 (merged) -
Update the PAM admin perms, so we can SSH into nodes.
Exit Criterion
-
Validate problem space -
Implement an appropriate fix
Edited by Tarun Khandelwal