Configuration audit of sidekiq K8s that includes all dependencies external to the cluster

This issue will look at all external to cluster dependencies of our sidekiq configuration to ensure we have configuration parity before migrating more queues the cluster.

gitlab.yml missing configuration:

• Object storage configuration

  • gitlab-gprd-artifacts ✅
  • gitlab-gprd-external-diffs ✅
  • gitlab-gprd-lfs-objects ✅
  • gitlab-gprd-uploads ✅
  • gitlab-gprd-package-repo ✅
  • gitlab-gprd-dependency-proxy ❌ Missing from K8s configuration gitlab-org/charts/gitlab#1284 (comment 352681437) . Not an issue until we migrate catch-all https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/10231#note_355206352
  • gitlab-gprd-terraform-state gitlab-com/gl-infra/k8s-workloads/gitlab-com!251 (merged)

• Sentry configuration - this section is missing in the k8s config - fixed with gitlab-org/charts/gitlab!1398 (merged)

  ## Error Reporting and Logging with Sentry
  sentry:
    enabled: true
    dsn: https://**:**@sentry.gitlab.net/2
    clientside_dsn: https://***@sentry.gitlab.net/15
    environment:  # e.g. development, staging, production

• Cron job config, why is schedule_migrate_external_diffs_worker different between k8s and vms? Every entry on VMs has a corresponding entry in K8s, I believe this is ok gitlab-com/gl-infra/k8s-workloads/gitlab-com!244 (merged)

K8s config:

  cron_jobs:
    admin_email_worker:
      cron: 0 0 * * 0
    expire_build_artifacts_worker:
      cron: 50 * * * *
    pages_domain_verification_cron_worker:
      cron: '*/15 * * * *'
    pipeline_schedule_worker:
      cron: '*/5 * * * *'
    pseudonymizer_worker:
      cron: 0 23 * * *
    repository_archive_cache_worker:
      cron: 30 * * * *
    repository_check_worker:
      cron: 20 * * * *
    schedule_migrate_external_diffs_worker:
      cron: 15 * * * *
    stuck_ci_jobs_worker:
      cron: 0 * * * *

VM config:

  cron_jobs:
    # Flag stuck CI builds as failed
    stuck_ci_jobs_worker:
      cron:
    # Remove expired build artifacts
    expire_build_artifacts_worker:
      cron:
    # Stop expired environments
    environments_auto_stop_cron_worker:
      cron:
    # Schedule pipelines in the near future
    pipeline_schedule_worker:
      cron: "*/5 * * * *"
    # Periodically run 'git fsck' on all repositories. If started more than
    # once per hour you will have concurrent 'git fsck' jobs.
    repository_check_worker:
      cron:
    # Send admin emails once a week
    admin_email_worker:
      cron:
    # Send emails about personal tokens about to expired
    personal_access_tokens_expiring_worker:
      cron:
    # Remove outdated repository archives
    repository_archive_cache_worker:
      cron: "30 * * * *"

    # Archive live traces which have not been archived yet
    ci_archive_traces_cron_worker:
      cron:

    # Verify custom GitLab Pages domains
    pages_domain_verification_cron_worker:
      cron:

    # Obtain and renew SSL certificates for pages domain through Let's Encrypt
    pages_domain_ssl_renewal_cron_worker:
      cron:

    # Removes unverified pages domains
    pages_domain_removal_cron_worker:
      cron:

    # Periodically migrate diffs from the database to external storage
    schedule_migrate_external_diffs_worker:
      cron: "*/5 * * * *"

• Repositories - path is set to /var/opt/gitlab/repo for all storages, probably ok

K8s config:

  repositories:
    storages:
      default:
        path: /var/opt/gitlab/repo
        gitaly_address: tcp://file-01-stor-gprd.c.gitlab-production.internal:9999
      nfs-file01:
        path: /var/opt/gitlab/repo
        gitaly_address: tcp://file-01-stor-gprd.c.gitlab-production.internal:9999
      ...

VM config:

storages: {"default":{"path":"/var/opt/gitlab/git-data-file01/repositories","gitaly_address":"tcp://file-01-stor-gprd.c.gitlab-production.internal:9999"},"nfs-file01":{"path":"/var/opt/gitlab/git-data-file01/repositories","gitaly_address":"tcp://file-01-stor-gprd.c.gitlab-production.internal:9999"} ...

• Pseudonymizer - missing configuraiton in K8s ❌ gitlab-com/gl-infra/k8s-workloads/gitlab-com!243 (merged) - We decided not to include this in K8s since it is deprecated and we aren't using it on gitlab.com

  ## Pseudonymizer settings
  pseudonymizer:
    manifest:
    upload:
      remote_directory: "gitlab-elt"
      connection: {"provider":"Google","google_project":"gitlab-analysis","google_json_key_location":"/etc/gitlab/meltano-gcs-creds.json"}