Gitaly internal calls don't work with the existing Ingress in OpenShift recommendations
Summary
Existing Ingress in OpenShift documentation guides user to use domain name associated with the Route created for Webservice for GitLab domain.
This configuration causes issue with internal Gitaly requests to GitLab domain - it redirects calls tp domain associated with OpenShift cluster {"kubernetes.io/service-name":"openshift-ingress/router-default"}
and not GitLab
This affects both cloud native hybrid setup (gitlab-org/quality/quality-engineering/team-tasks#749 (comment 1286686416)) and full cloud native setup (#1085 (comment 1291964366))
Identified fix work
-
Adding A record to private zone fixed the issue in CNH - #1087 (comment 1306088811) -
to investigate if Private zone change fixes issue in OS CI cluster -
update documentation at https://docs.gitlab.com/operator/openshift_ingress.html#configuration to also create A records in private DNS -
potentially add a note to docs that it's preferable to run GitLab on a separate domain than OS cluster ?
Steps to reproduce
Follow https://docs.gitlab.com/operator/openshift_ingress.html#configuration to configure GitLab domain in OpenShift
Configuration used
- gitlab-org/quality/quality-engineering/team-tasks#749 (comment 1254071523)
- or CI OpenShift cluster in case of #1085 (comment 1291969961)
Current behavior
Gitaly internal calls to GitLab are directed to OpenShift cluster
Expected behavior
Gitaly internal calls to GitLab work or documentation Ingress in OpenShift updated to make the existing setup work.
Versions
- Operator: latest
- Platform:
- Self-hosted: OpenShift
Relevant logs
Please see details in gitlab-org/quality/quality-engineering/team-tasks#749 (comment 1286686416)
root@openshift-hybrid-nish-gitaly-1:/# host gitlab.apps.qe-os-cnh-mvc.k8s-ft.win
gitlab.apps.qe-os-cnh-mvc.k8s-ft.win has address 34.67.140.96
root@openshift-hybrid-nish-gitaly-1:/# host gitlab.apps.qe-os-cnh-mvc.k8s-ft.win 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:
gitlab.apps.qe-os-cnh-mvc.k8s-ft.win has address 34.28.112.235
34.67.140.96
- is the IP associated with*.apps.qe-os-cnh-mvc
from MVC DNS records which appears to be created during OpenShift cluster creation. When looking at this IP in GCP and its forwarding rule - it says{"kubernetes.io/service-name":"openshift-ingress/router-default"}
.