Skip to content

Resolve "Add support for multiple virtual_storages in Praefect"

Mitchell Nielsen requested to merge 2307-praefect-support-multiple-virtual-storages into master

What does this MR do?

Functionality requirements thread: #2307 (comment 436275383)

Related issues

Closes #2307 (closed)

Previous Praefect MR: !1493 (merged)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • Merge Request Title and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Tests added
  • Integration tests added to GitLab QA
  • Equivalent MR/issue for omnibus-gitlab opened

Closes #2307 (closed)

To do

  • Confirm approach is valid via feedback (clarifying "definition of done")
  • Add Rspec tests
  • Update documentation

Testing

The following values are used for testing:

global:
  gitaly:
    tls:
      enabled: true
  praefect:
    enabled: true
    tls:
      enabled: true
      secretName: praefect-tls
    virtualStorages:
    - name: default
      gitalyReplicas: 1
      maxUnavailable: 0
      tlsSecretName: gitaly-default-tls
    - name: vs2
      gitalyReplicas: 1
      maxUnavailable: 0
      tlsSecretName: gitaly-vs2-tls
  operator:
    enabled: true # to test the Gitaly pause job

For the scenarios where Praefect isn't used, the YAML above was simply commented out/not referenced.

The following docs were also referenced:

Certificates were created like so:

$ mkdir certs
$ cd certs
$ NAMESPACE=gitlab ../scripts/generate_certificates.sh gitaly-default
$ NAMESPACE=gitlab ../scripts/generate_certificates.sh gitaly-vs2
$ NAMESPACE=gitlab ../scripts/generate_certificates.sh praefect
$ kubectl -n gitlab create secret tls gitaly-default-tls --cert=./gitaly-default.crt --key=./gitaly-default.key
$ kubectl -n gitlab create secret tls gitaly-vs2-tls --cert=./gitaly-vs2.crt --key=./gitaly-vs2.key
$ kubectl -n gitlab create secret tls praefect-tls --cert=./praefect.crt --key=./praefect.key

Pods

When using Praefect with 2 virtual storages

❯ k get pods -n gitlab -l 'app in (gitaly-default, gitaly-vs2, praefect)'
NAME                      READY   STATUS    RESTARTS   AGE
gitlab-gitaly-default-0   1/1     Running   0          12m
gitlab-gitaly-default-1   1/1     Running   0          13m
gitlab-gitaly-default-2   1/1     Running   0          14m
gitlab-gitaly-vs2-0       1/1     Running   0          15m
gitlab-gitaly-vs2-1       1/1     Running   0          15m
gitlab-praefect-0         1/1     Running   0          4m59s
gitlab-praefect-1         1/1     Running   0          6m6s

When not using Praefect

❯ k get pods -n gitlab -l 'app in (praefect, gitaly-default)'
NAME                      READY   STATUS    RESTARTS   AGE
gitlab-gitaly-default-0   1/1     Running   0          51m

Gitaly status check

git@gitlab-gitaly-coldstorage-0:/$ gitaly-hooks check /etc/gitaly/config.toml
Checking GitLab API access: OK
GitLab version: 13.6.0-pre
GitLab revision:
GitLab Api version: v4
Redis reachable for GitLab: true
OK

With adjusted weights

Go to $GITLAB_URL/admin/application_settings/repository > Repository storage and assign:

  • 0 weight to the default storage
  • 100 weight to the vs2 storage

Then, create a new project and confirm there are no errors. You can also confirm that the repository data is on the vs2 Gitaly nodes under /home/git/repositories.

Edited by Mitchell Nielsen

Merge request reports