## Context The container registry supports webhook notifications as described in https://docs.docker.com/registry/notifications/. This is currently used for delivering notifications to Rails whenever an image is pushed, but the information is only used to feed metrics (push count). Related to https://gitlab.com/gitlab-org/gitlab/-/issues/21042#note_542630004. ## Problem The as-is notification mechanism does not offer delivery guarantees, as notifications are staged in vulnerable per-instance in-memory queues. Because of this, it's risky to use these async notifications for any kind of data synchronization, as data inconsistencies may occur. ## Solution In order to provide the at-least-once delivery guarantee, we should use a reliable and persistent queueing system to store these messages and retry their delivery in case of failures or instance restarts. This can be achieved by using either the metadata database or a separate store (e.g. Redis) to store and process messages from. ## Helpful links - Previous discussion about this topic: https://gitlab.com/gitlab-org/gitlab/-/issues/20378#note_287587313