Please review whether registry notifier is needed, and remove if not

Summary

The GitLab Registry chart configuration is set up to send a notification to the primary API url. I've been made aware of a discussion where it was pointed out that this isn't needed, and may not actually be desired at all.

Steps to reproduce

(Please provide the steps to reproduce the issue)

Configuration used

Default template code: https://gitlab.com/gitlab-org/charts/gitlab/-/blob/3a509377bdf8be0dccf3fc91286d37cfa51fd7dd/templates/_registry.tpl#L57-73

geo:
  enabled: true
  nodeName: REDACTED
  psql:
    host: REDACTED
    password:
      key: password
      secret: REDACTED
    port: "5432"
  registry:
    replication:
      enabled: true
      primaryApiUrl: https://gitlab.example.com
  role: secondary

Current behavior

A Registry in a secondary site is sending notifications to the primary site, and getting 401s.

Expected behavior

A Registry in a secondary site should not send notifications.

(Note that a Registry in a primary site should send notifications to the primary site.)

Versions

• Chart: (tagged version | branch | hash git rev-parse HEAD)
• Platform:
  • Cloud: (GKE | AKS | EKS | ?)
  • Self-hosted: (OpenShift | Minikube | Rancher RKE | ?)
• Kubernetes: (kubectl version)
  • Client:
  • Server:
• Helm: (helm version)
  • Client:
  • Server:

Relevant logs

2025-01-13T21:21:42.516474456Z {"level":"error","msg":"retryingsink: error writing events: httpSink{https://gitlab.example.com/api/v4/container_registry_event/events}: response status 401 Unauthorized unaccepted, retrying","time":"2025-01-13T21:21:42.516Z"}
2025-01-13T21:21:42.516493545Z {"level":"warning","msg":"httpSink{https://gitlab.example.com/api/v4/container_registry_event/events} encountered too many errors, backing off","time":"2025-01-13T21:21:42.516Z"}

(Please provide any relevate log snippets you have collected, using code blocks (```) to format)

added typebug label

Thanks @cody!

To expand on Expected behavior:

1. a Registry in a primary site should send notifications to the primary site
2. a Registry in a secondary site should not send notifications

The purpose of 1. is to make the primary site aware of all pushes, so that it can create a Geo event, which will propagate to secondary sites and cause them to pull the changes.

I think if a secondary site were to send notifications to the primary site, then 1 push will end up being pulled by a secondary site twice. Because a sync job that finds new changes will do a push to the secondary site's Registry.

---

The logs above show 401s. I assume the logs are from a secondary site (since the config has role: secondary) and the notification requests are truly not properly authenticated, perhaps because the key or secret is wrong?

---

The behavior of this template depends on the values of the variables inside of the primary site vs the secondary site:

{{/*
When Geo + Container Registry syncing enabled, adds the following notifier
*/}}
{{- define "global.geo.registry.syncNotifier" -}}
{{- if and .Values.global.geo.enabled .Values.global.geo.registry.replication.enabled -}}
endpoints:
  - name: geo_event
    url: https://{{ include "gitlab.gitlab.hostname" . }}/api/v4/container_registry_event/events
    timeout: 2s
    {{/* DEPRECATED: use maxretries instead https://gitlab.com/gitlab-org/container-registry/-/issues/1243.*/}}
    threshold: 5
    maxretries: 0
    backoff: 1s
    headers:
      Authorization:
        secret: {{ template "gitlab.registry.notificationSecret.secret" $ }}
        key: {{ template "gitlab.registry.notificationSecret.key" $ }}
{{- else -}}
endpoints: []
{{- end -}}
{{- end -}}

In your set up, could you show what these values evaluate to, for each site in the environment?

I see this section says:

Define this in the primary site configs only.

Which looks like it conflicts with the config in the description.

This is really confusing when combined with GitLab Linux Package docs because that doc configures Registry notifications more directly, on the primary site. But it configures geo_registry_replication_enabled and geo_registry_replication_primary_api_url on the secondary site. Whereas charts tells you to define those only in the primary site.

workflowproblem validation: Even if the code is "ok", the docs are not, because they are confusing. Someone following https://docs.gitlab.com/charts/advanced/geo/index.html#registry should be able to set up Container Registry replication without a lot of confusion. It is also possible that the code is not ideal; I am not sure.

workflowsolution validation: We need to set up an environment with Chart + Geo (or work with someone who has one) and then tinker with it to determine:

1. What the primary and secondary site configs should look like (and mention both to the sparse Registry replication setup doc section)
2. How the Registry template should work (hopefully it is fine as-is)
3. How the docs should be modified to make the most sense (to at least resolve this confusion)

Marking severity3 since this does not seem to block replication.

With the testing + additional documentation/coding work, would you think this is about a weight 3 @mkozono ?

Yes I think weight 3 is good, thanks.

added bugfunctional groupgeo workflowsolution validation labels

added devopstenant scale sectioninfrastructure platforms labels

added documentation severity3 labels

changed the description

set weight to 3

changed milestone to %Backlog