Update documentation to match Gitaly's new configuration structure

What does this MR do?

Omnibus previously had a custom configuration structure for Gitaly that didn't match the actual structure of Gitaly. This made a number of things more difficult than they had to be. Omnibus was recently updated to place Gitaly's configuration under gitaly['configuration'] hash which gets directly marshaled as TOML for the final Gitaly configuration. While there is still backwards compatibility handling in place, we should update the documentation to use the new structure in preparation for deprecating the old configuration keys. This commit updates Gitaly's documentation to refer to the new keys.

Related issues

gitaly#4467 (closed)
gitaly#4754 (closed)
omnibus-gitlab!6621 (merged)

Author's checklist

• Optional. Consider taking the GitLab Technical Writing Fundamentals course.
• Follow the:
  • Documentation process.
  • Documentation guidelines.
  • Style Guide.
• If you're adding or changing the main heading of the page (H1), ensure that the product tier badge is added.
• If you are a GitLab team member, request a review based on:
  • The documentation page's metadata.
  • The associated Technical Writer.

If you are a GitLab team member and only adding documentation, do not add any of the following labels:

• ~"frontend"
• ~"backend"
• ~"type::bug"
• ~"database"

These labels cause the MR to be added to code verification QA issues.

Reviewer's checklist

Documentation-related MRs should be reviewed by a Technical Writer for a non-blocking review, based on Documentation Guidelines and the Style Guide.

If you aren't sure which tech writer to ask, use roulette or ask in the #docs Slack channel.

• If the content requires it, ensure the information is reviewed by a subject matter expert.
• Technical writer review items:
  • Ensure docs metadata is present and up-to-date.
  • Ensure the appropriate labels are added to this MR.
  • Ensure a release milestone is set.
  • If relevant to this MR, ensure content topic type principles are in use, including:
    • The headings should be something you'd do a Google search for. Instead of Default behavior, say something like Default behavior when you close an issue.
    • The headings (other than the page title) should be active. Instead of Configuring GDK, say something like Configure GDK.
    • Any task steps should be written as a numbered list.
    • If the content still needs to be edited for topic types, you can create a follow-up issue with the docs-technical-debt label.
• Review by assigned maintainer, who can always request/require the reviews above. Maintainer's review can occur before or after a technical writer review.

doc/administration/gitaly/configure_gitaly.md

@@ -143,7 +143,13 @@ Gitaly and GitLab use two shared secrets for authentication:
 To configure the _Gitaly token_, edit `/etc/gitlab/gitlab.rb`:
 
    ```ruby
-   gitaly['auth_token'] = 'abc123secret'
+   gitaly['configuration'] = {
+      # ...
+      auth: {
+        # ...
+        token: 'abc123secret',
+      },
+   }
    ```
 
 There are two ways to configure the _GitLab Shell token_.
@@ -230,14 +236,21 @@ Updates to example must be made at:
    # Don't forget to copy `/etc/gitlab/gitlab-secrets.json` from Gitaly client to Gitaly server.
    gitlab_rails['internal_api_url'] = 'https://gitlab.example.com'
 
-   # Make Gitaly accept connections on all network interfaces. You must use
-   # firewalls to restrict access to this address/port.
-   # Comment out following line if you only want to support TLS connections
-   gitaly['listen_addr'] = "0.0.0.0:8075"
-
-   # Authentication token to ensure only authorized servers can communicate with
-   # Gitaly server
-   gitaly['auth_token'] = 'AUTH_TOKEN'
+   gitaly['configuration'] = {
+      # ...
+      #
+      # Make Gitaly accept connections on all network interfaces. You must use
+      # firewalls to restrict access to this address/port.
+      # Comment out following line if you only want to support TLS connections
+      listen_addr: '0.0.0.0:8075',
+      auth: {
+        # ...
+        #
+        # Authentication token to ensure only authorized servers can communicate with
+        # Gitaly server
+        token: 'AUTH_TOKEN',
+      },
+   }
    ```
 
 1. Append the following to `/etc/gitlab/gitlab.rb` for each respective Gitaly server:
@@ -247,24 +260,33 @@ Updates to example must be made at:
    On `gitaly1.internal`:
 
    ```ruby
-   git_data_dirs({
-     'default' => {
-       'path' => '/var/opt/gitlab/git-data'
-     },
-     'storage1' => {
-       'path' => '/mnt/gitlab/git-data'
-     },
-   })
+   gitaly['configuration'] = {
+      # ...
+      storage: [
+         {
+            name: 'default',
+            path: '/var/opt/gitlab/git-data',
+         },
+         {
+            name: 'storage1',
+            path: '/mnt/gitlab/git-data',
+         },
+      ],
+   }
    ```
 
    On `gitaly2.internal`:
 
    ```ruby
-   git_data_dirs({
-     'storage2' => {
-       'path' => '/srv/gitlab/git-data'
-     },
-   })
+   gitaly['configuration'] = {
+      # ...
+      storage: [
+         {
+            name: 'storage2',
+            path: '/srv/gitlab/git-data',
+         },
+      ],
+   }
    ```
 
 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
@@ -461,17 +483,28 @@ example:
 git_data_dirs({
   'default' => { 'gitaly_address' => 'tcp://gitaly1.internal:8075' },
   # Address of the GitLab server that also has Gitaly running on it
-  'storage1' => { 'gitaly_address' => 'tcp://gitlab.internal:8075', 'path' => '/mnt/gitlab/git-data' },
+  'storage1' => { 'gitaly_address' => 'tcp://gitlab.internal:8075' },
   'storage2' => { 'gitaly_address' => 'tcp://gitaly2.internal:8075' },
 })
 
-# Make Gitaly accept connections on all network interfaces
-gitaly['listen_addr'] = "0.0.0.0:8075"
-
-# Or for TLS
-gitaly['tls_listen_addr'] = "0.0.0.0:9999"
-gitaly['certificate_path'] = "/etc/gitlab/ssl/cert.pem"
-gitaly['key_path'] = "/etc/gitlab/ssl/key.pem"
+gitaly['configuration'] = {
+  # ...
+  #
+  # Make Gitaly accept connections on all network interfaces
+  listen_addr: '0.0.0.0:8075',
+  # Or for TLS
+  tls_listen_addr: '0.0.0.0:9999',
+  tls: {
+    certificate_path:  '/etc/gitlab/ssl/cert.pem',
+    key_path: '/etc/gitlab/ssl/key.pem',
+  },
+  storage: [
+    {
+      name: 'storage1',
+      path: '/mnt/gitlab/git-data',
+    },
+  ],
+}
 ```
 
 `path` can be included only for storage shards on the local Gitaly server.
@@ -600,16 +633,21 @@ To configure Gitaly with TLS:
    <!-- Updates to following example must also be made at https://gitlab.com/gitlab-org/charts/gitlab/blob/master/doc/advanced/external-gitaly/external-omnibus-gitaly.md#configure-omnibus-gitlab -->
 
    ```ruby
-   gitaly['tls_listen_addr'] = "0.0.0.0:9999"
-   gitaly['certificate_path'] = "/etc/gitlab/ssl/cert.pem"
-   gitaly['key_path'] = "/etc/gitlab/ssl/key.pem"
+   gitaly['configuration'] = {
+      # ...
+      tls_listen_addr: '0.0.0.0:9999',
+      tls: {
+        certificate_path: '/etc/gitlab/ssl/cert.pem',
+        key_path: '/etc/gitlab/ssl/key.pem',
+      },
+   }
    ```
 
 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
 1. Verify Gitaly traffic is being served over TLS by
    [observing the types of Gitaly connections](#observe-type-of-gitaly-connections).
 1. Optional. Improve security by:
-   1. Disabling non-TLS connections by commenting out or deleting `gitaly['listen_addr']` in
+   1. Disabling non-TLS connections by commenting out or deleting `gitaly['configuration][:listen_addr]` in
       `/etc/gitlab/gitlab.rb`.
    1. Saving the file.
    1. [Reconfiguring GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
@@ -733,9 +771,16 @@ settings:
 1. Edit `/etc/gitlab/gitlab.rb`:
 
    ```ruby
-   # Default is 2 workers. The minimum is 2; 1 worker is always reserved as
-   # a passive stand-by.
-   gitaly['ruby_num_workers'] = 4
+   gitaly['configuration'] = {
+      # ...
+      'gitaly-ruby': {
+        # ...
+        #
+        # Default is 2 workers. The minimum is 2; 1 worker is always reserved as
+        # a passive stand-by.
+        num_workers: 4
+      },
+   }
    ```
 
 1. Save the file, and then [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
@@ -775,21 +820,23 @@ example:
 
 ```ruby
 # in /etc/gitlab/gitlab.rb
-
-gitaly['concurrency'] = [
-  {
-    'rpc' => "/gitaly.SmartHTTPService/PostUploadPackWithSidechannel",
-    'max_per_repo' => 20,
-    'max_queue_time' => "1s",
-    'max_queue_size' => 10
-  },
-  {
-    'rpc' => "/gitaly.SSHService/SSHUploadPackWithSidechannel",
-    'max_per_repo' => 20
-    'max_queue_time' => "1s",
-    'max_queue_size' => 10
-  }
-]
+gitaly['configuration'] = {
+   # ...
+   concurrency: [
+      {
+         rpc: '/gitaly.SmartHTTPService/PostUploadPackWithSidechannel',
+         max_per_repo: 20,
+         max_queue_time: '1s',
+         max_queue_size: 10,
+      },
+      {
+         rpc: '/gitaly.SSHService/SSHUploadPackWithSidechannel',
+         max_per_repo: 20,
+         max_queue_time: '1s',
+         max_queue_size: 10,
+      },
+   ],
+}
 ```
 
 - `rpc` is the name of the RPC to set a concurrency limit for per repository.
@@ -864,40 +911,47 @@ When these limits are reached, performance may be reduced and users may be disco
 > This method of configuring repository cgroups was introduced in GitLab 15.1.
 
 To configure repository cgroups in Gitaly using the new method, use the following settings for the new configuration method
-to `gitaly['cgroups']` in `/etc/gitlab/gitlab.rb`:
+to `gitaly['configuration'][:cgroups]` in `/etc/gitlab/gitlab.rb`:
 
-- `cgroups_mountpoint` is where the parent cgroup directory is mounted. Defaults to `/sys/fs/cgroup`.
-- `cgroups_hierarchy_root` is the parent cgroup under which Gitaly creates groups, and
+- `mountpoint` is where the parent cgroup directory is mounted. Defaults to `/sys/fs/cgroup`.
+- `hierarchy_root` is the parent cgroup under which Gitaly creates groups, and
    is expected to be owned by the user and group Gitaly runs as. Omnibus GitLab
    creates the set of directories `mountpoint/<cpu|memory>/hierarchy_root`
    when Gitaly starts.
-- `cgroups_memory_bytes` is the total memory limit that is imposed collectively on all
+- `memory_bytes` is the total memory limit that is imposed collectively on all
    Git processes that Gitaly spawns. 0 implies no limit.
-- `cgroups_cpu_shares` is the CPU limit that is imposed collectively on all Git
+- `cpu_shares` is the CPU limit that is imposed collectively on all Git
    processes that Gitaly spawns. 0 implies no limit. The maximum is 1024 shares,
    which represents 100% of CPU.
-- `cgroups_repositories_count` is the number of cgroups in the cgroups pool. Each time a new Git
+- `repositories.count` is the number of cgroups in the cgroups pool. Each time a new Git
   command is spawned, Gitaly assigns it to one of these cgroups based
   on the repository the command is for. A circular hashing algorithm assigns
   Git commands to these cgroups, so a Git command for a repository is
   always assigned to the same cgroup.
-- `cgroups_repositories_memory_bytes` is the total memory limit imposed on all Git processes contained in a repository cgroup.
-  0 implies no limit. This value cannot exceed that of the top level `cgroups_memory_bytes`.
-- `cgroups_repositories_cpu_shares` is the CPU limit that is imposed on all Git processes contained in a repository cgroup.
+- `repositories.memory_bytes` is the total memory limit imposed on all Git processes contained in a repository cgroup.
+  0 implies no limit. This value cannot exceed that of the top level `memory_bytes`.
+- `repositories.cpu_shares` is the CPU limit that is imposed on all Git processes contained in a repository cgroup.
   0 implies no limit. The maximum is 1024 shares, which represents 100% of CPU.
-  This value cannot exceed that of the top level`cgroups_cpu_shares`.
+  This value cannot exceed that of the top level`cpu_shares`.
 
 For example:
 
 ```ruby
 # in /etc/gitlab/gitlab.rb
-gitaly['cgroups_mountpoint'] = "/sys/fs/cgroup"
-gitaly['cgroups_hierarchy_root'] => "gitaly"
-gitaly['cgroups_memory_bytes'] = 64424509440,  # 60gb
-gitaly['cgroups_cpu_shares'] = 1024
-gitaly['cgroups_repositories_count'] => 1000,
-gitaly['cgroups_repositories_memory_bytes'] => 32212254720 # 20gb
-gitaly['cgroups_repositories_cpu_shares'] => 512
+gitaly['configuration'] = {
+  # ...
+  cgroups: {
+    mountpoint: '/sys/fs/cgroup',
+    hierarchy_root: 'gitaly',
+    memory_bytes: 64424509440, # 60gb
+    cpu_shares: 1024,
+    repositories: {
+      count: 1000,
+      memory_bytes: 32212254720, # 20gb
+      cpu_shares: 512,
+    },
+  },
+}
 ```
 
 ### Configure repository cgroups (legacy method)
@@ -968,10 +1022,16 @@ Make sure to schedule this during off-peak hours and keep the duration short (fo
 Edit `/etc/gitlab/gitlab.rb` and add:
 
 ```ruby
-gitaly['daily_maintenance_start_hour'] = 4
-gitaly['daily_maintenance_start_minute'] = 30
-gitaly['daily_maintenance_duration'] = '30m'
-gitaly['daily_maintenance_storages'] = ["default"]
+gitaly['configuration'] = {
+  # ...
+  daily_maintenance: {
+    # ...
+    start_hour: 4,
+    start_minute: 30,
+    duration: '30m',
+    storages: ['default'],
+  },
+}
 ```
 
 **For installations from source**
@@ -1018,7 +1078,13 @@ transitioning" mode as follows:
 
 ```ruby
 # in /etc/gitlab/gitlab.rb
-gitaly['auth_transitioning'] = true
+gitaly['configuration'] = {
+  # ...
+  auth: {
+    # ...
+    transitioning: true,
+  },
+}
 ```
 
 After you have made this change, your [Prometheus query](#verify-authentication-monitoring)
@@ -1038,8 +1104,13 @@ To update to a new Gitaly authentication token, on each Gitaly client **and** Gi
 
    ```ruby
    # in /etc/gitlab/gitlab.rb
-
-   gitaly['auth_token'] = '<new secret token>'
+   gitaly['configuration'] = {
+      # ...
+      auth: {
+         # ...
+         token: '<new secret token>',
+      },
+   }
    ```
 
 1. Restart Gitaly:
@@ -1069,7 +1140,13 @@ your Gitaly servers as follows:
 
 ```ruby
 # in /etc/gitlab/gitlab.rb
-gitaly['auth_transitioning'] = false
+gitaly['configuration'] = {
+  # ...
+  auth: {
+    # ...
+    transitioning: false,
+  },
+}
 ```
 
 WARNING:
@@ -1147,9 +1224,15 @@ setting is discussed in greater detail below.
 In `/etc/gitlab/gitlab.rb`, set:
 
 ```ruby
-gitaly['pack_objects_cache_enabled'] = true
-## gitaly['pack_objects_cache_dir'] = '/var/opt/gitlab/git-data/repositories/+gitaly/PackObjectsCache'
-## gitaly['pack_objects_cache_max_age'] = '5m'
+gitaly['configuration'] = {
+  # ...
+  pack_objects_cache: {
+    # ...
+    enabled: true,
+    # dir: '/var/opt/gitlab/git-data/repositories/+gitaly/PackObjectsCache',
+    # max_age: '5m',
+  },
+}
 ```
 
 #### `enabled` defaults to `false`
@@ -1315,6 +1398,31 @@ process repositories that do not pass consistency checks.
 For Omnibus GitLab installations, edit `/etc/gitlab/gitlab.rb` and set the
 following keys (in this example, to disable the `hasDotgit` consistency check):
 
+- In [GitLab 15.9](https://gitlab.com/gitlab-org/gitaly/-/issues/4754) and later:
+
+  ```ruby
+  ignored_blobs = "/etc/gitlab/instance_wide_ignored_git_blobs.txt"
+
+  gitaly['configuration'] = {
+    # ...
+    git: {
+      # ...
+      config: [
+        # Populate a file with one unabbreviated SHA-1 per line.
+        # See https://git-scm.com/docs/git-config#Documentation/git-config.txt-fsckskipList
+        { key: "fsck.skipList", value: ignored_blobs },
+        { key: "fetch.fsck.skipList", value: ignored_blobs },
+        { key: "receive.fsck.skipList", value: ignored_blobs },
+
+        { key: "fsck.hasDotgit", value: "ignore" },
+        { key: "fetch.fsck.hasDotgit", value: "ignore" },
+        { key: "receive.fsck.hasDotgit", value: "ignore" },
+        { key: "fsck.missingSpaceBeforeEmail", value: "ignore" },
+      ],
+    },
+  }
+  ```
+
 - In [GitLab 15.3](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6800) and later:
 
   ```ruby
@@ -1423,7 +1531,13 @@ proposed in issue [19185](https://gitlab.com/gitlab-org/gitlab/-/issues/19185).
 1. Edit `/etc/gitlab/gitlab.rb` and configure `gitaly['gpg_signing_key_path']`:
 
    ```ruby
-   gitaly['gpg_signing_key_path'] = "/etc/gitlab/gitaly/signing_key.gpg"
+   gitaly['configuration'] = {
+      # ...
+      git: {
+        # ...
+        signing_key: '/etc/gitlab/gitaly/signing_key.gpg',
+      },
+   }
    ```
 
 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).