[Geo] Docker image replication(2/4)
What does this MR do?
This MR adds Docker registry sync to a secondary node. The basic principle is that same as for files and git repositories. When we push a new image to the docker registry the notification event is created in the primary database and it will be propagated to the secondary. We intentionally don't track every pushed tag, instead, a secondary node makes a request to the primary's Registry API to get the list of available tags (with their digests) in this particular docker registry repository and with that information, the secondary pulls new images or removes outdated ones.
Why we don’t track updates on per image basis?
Because it creates some problems on the primary node, it will be needed to track all those tags and it will be also needed to actualize that information periodically because the docker registry notification system is not reliable at all. Also, that would require to maintain a huge database table and make SQL against it often so it’s unpractical.
Are there points in the code the reviewer needs to double check?
Why was this MR needed?
Screenshots (if relevant)
Does this MR meet the acceptance criteria?
-
Changelog entry added, if necessary -
Documentation created/updated https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3632/diffs -
API support added -
Tests added for this feature/bug - Review
-
Has been reviewed by UX -
Has been reviewed by Frontend -
Has been reviewed by Backend -
Has been reviewed by Database
-
-
Conform by the merge request performance guides -
Conform by the style guides -
Squashed related commits together -
Internationalization required/considered -
If paid feature, have we considered GitLab.com plan and how it works for groups and is there a design for promoting it to users who aren't on the correct plan
What are the relevant issue numbers?
https://gitlab.com/gitlab-org/gitlab-ee/issues/2870
TODO:
-
Registry for containers -
Setup events from registry -
Implement even system for containers -
Implement image sync -
Implement sync for backfill condition (Dispatch worker) -
Data model (tables, foreign keys) -
GeoNodeStatus parameters calculation -
Send MR to early review and fix comments -
GeoNodeStatus parameters are shown in UI, rake tasks, API -
GeoNodeStatus parameters are exposed in Prometheus -
Notification has to be enabled when Geo is enabled -
Turn off/on -
Specs for everything -
Add Lease for sync service -
Admin documentation (separate MR) -
Authentication in lib/api/container_registry_event.rb