Skip to content

Add support for Geo Unified URL feature

Clemens Beck requested to merge add-support-for-geo-unified-url into master

What does this MR do?

Adaptions to make Unified URL work with the GitLab Helm chart.

1️⃣ Extra Ingress Controller

Traffic between Geo sites has X-Forwarded-* headers set. By default, NGINX modifies these headers before forwarding the traffic. This can be disabled controller wide with use-forwarded-headers.

To avoid modifying headers of other services, a second NGINX subchart has been added.

2️⃣ Extra Ingress

Allows to configure an additional webservice Ingress that can be used to accept internal traffic. It can be configured independently of the default webservice Ingress.

3️⃣ GitLab host:

Geo secondary sites now use the external/unified hostname in gitlab.yml by specifying it in global.hosts.

Related issues

Closes #4532 (closed)

Relates #3845 (closed) (spike)

Test plan

Prerequisites:

  • 2 K8s Clusters

  • a (sub) domain

  • a GitLab Premium or Ultimate Developer License

  • two DB VM's (Alternative: Pods + a NodePort or LoadBalancer service, but the primary DB must be accessible from the secondary DB node)

    primary.db.yaml 
    apiVersion: v1
kind: Pod
metadata:
  name: primary-geo-db
  labels:
    app.kubernetes.io/name: geo-primary-db
spec:
  containers:
  - name: primary-geo-db
    image: gitlab/gitlab-ee:16.1.1-ee.0
    ports:
    - containerPort: 5432
    env:
      - name: GITLAB_OMNIBUS_CONFIG
        value: |
          ### Geo Primary
          external_url '<external URL>'
          roles ['geo_primary_role']
          # The unique identifier for the Geo node.
          gitlab_rails['geo_node_name'] = 'Primary Node'
          gitlab_rails['auto_migrate'] = false
          ## turn off everything but the DB
          sidekiq['enable']=false
          puma['enable']=false
          gitlab_workhorse['enable']=false
          nginx['enable']=false
          geo_logcursor['enable']=false
          grafana['enable']=false
          gitaly['enable']=false
          redis['enable']=false
          prometheus_monitoring['enable'] = false
          kas['enable']=false
          ## Configure the DB for network
          postgresql['enable'] = true
          postgresql['listen_address'] = '0.0.0.0'
          postgresql['sql_user_password'] = '<replace>'
          # !! CAUTION !!
          # This list of CIDR addresses should be customized
          # - primary application deployment
          # - secondary database node(s)
          postgresql['md5_auth_cidr_addresses'] = ['0.0.0.0/0']
    volumeMounts:
      - mountPath: /var/opt/gitlab
        name: data
      - mountPath: /etc/gitlab
        name: config
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: primary-db-data
    - name: config
      persistentVolumeClaim:
        claimName: primary-db-config
---
apiVersion: v1
kind: Service
metadata:
  name: primdb
spec:
  selector:
    app.kubernetes.io/name: geo-primary-db
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: primary-db-data
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2G
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: primary-db-config
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100M

    secondary.db.yaml 
    apiVersion: v1
kind: Pod
metadata:
  name: secondary-geo-db
  labels:
    app.kubernetes.io/name: geo-secondary-db
spec:
  containers:
  - name: secondary-geo-db
    image: gitlab/gitlab-ee:16.1.1-ee.0
    ports:
    - name: pg
      containerPort: 5432
    - name: geo
      containerPort: 5431
    env:
      - name: GITLAB_OMNIBUS_CONFIG
        value: |
          ### Geo Secondary
          # external_url must match the Primary cluster's external_url
          external_url '<external url>'
          roles ['geo_secondary_role']
          gitlab_rails['enable'] = true
          # The unique identifier for the Geo node.
          gitlab_rails['geo_node_name'] = 'Secondary Node'
          gitlab_rails['auto_migrate'] = false
          geo_secondary['auto_migrate'] = false
          ## turn off everything but the DB
          sidekiq['enable']=false
          puma['enable']=false
          gitlab_workhorse['enable']=false
          nginx['enable']=false
          geo_logcursor['enable']=false
          grafana['enable']=false
          gitaly['enable']=false
          redis['enable']=false
          prometheus_monitoring['enable'] = false
          ## Configure the DBs for network
          postgresql['enable'] = true
          postgresql['listen_address'] = '0.0.0.0'
          postgresql['sql_user_password'] = '<replace>'
          # !! CAUTION !!
          # This list of CIDR addresses should be customized
          # - secondary application deployment
          # - secondary database node(s)
          postgresql['md5_auth_cidr_addresses'] = ['0.0.0.0/0']
          geo_postgresql['listen_address'] = '0.0.0.0'
          geo_postgresql['sql_user_password'] = '<replace>'
          # !! CAUTION !!
          # This list of CIDR addresses should be customized
          # - secondary application deployment
          # - secondary database node(s)
          geo_postgresql['md5_auth_cidr_addresses'] = ['0.0.0.0/0']
          gitlab_rails['db_password']='<replace>'
    volumeMounts:
      - mountPath: /var/opt/gitlab
        name: data
      - mountPath: /etc/gitlab
        name: config
  volumes:
    - name: data
      persistentVolumeClaim:
        claimName: secondary-db-data
    - name: config
      persistentVolumeClaim:
        claimName: secondary-db-config
---
apiVersion: v1
kind: Service
metadata:
  name: secdb
spec:
  selector:
    app.kubernetes.io/name: geo-secondary-db
  ports:
    - name: pg
      protocol: TCP
      port: 5432
      targetPort: 5432
    - name: geo
      protocol: TCP
      port: 5431
      targetPort: 5431
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: secondary-db-data
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2G
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: secondary-db-config
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100M

Follow the updated doc/advanced/geo/index.md to create a unified URL Geo setup. To switch between Primary/Secondary you can change the external DNS record. Setting up DNS53 (or a similar service) is not required.

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • Merge Request Title and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com
  • When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Tests added
  • Integration tests added to GitLab QA
  • Equivalent MR/issue for omnibus-gitlab opened
  • Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.
Edited by Clemens Beck

Merge request reports