Skip to content

Trigger ProjectStatistics update on ContainerRegistry Event

Vijay Hawoldar requested to merge vij-container-reg-stats-3 into master

What does this MR do and why?

So that we can keep a Project's project_statistics#container_registry_size up to date, this MR triggers an update via an async job to update the statistics whenever a ContainerRegistry::Event for a push or delete action occurs.

Using the cache worker we can ensure deduplication of the jobs in the scenario where multiple events are fired in quick succession.

Refs #354480 (closed)

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

  1. Optional: (for step 7) Modify Project#container_repositories_size to return a number and bypass the .com checks etc, e.g.
      def container_repositories_size
        12345
      end
  2. Enable the feature flag container_registry_project_statistics:
      Feature.enable(:container_registry_project_statistics)
  3. Fire up the sidekiq dashboard so you can monitor jobs being enqueued, e.g. http://gdk.test:3000/admin/sidekiq/scheduled
  4. Ensure your config/gitlab.yml has a notification_secret set under the registry key, e.g.
      registry:
        notification_secret: test
  5. Hit the ContainerRegistry Events API endpoint using that secret as the Authorization value, and passing events in the body (change events.target.repository to be the full path of one of your projects):
    curl --request POST \
     --url http://gdk.test:3000/api/v4/container_registry_event/events \
     --header 'Authorization: test' \
     --header 'content-type: application/vnd.docker.distribution.events.v1+json' \
     --data '{
     "events": [
       {
             "action": "push",
             "target": {
               "tag": "latest",
               "repository": "group-a/project-a"
             }
           },
       {
             "action": "push",
             "target": {
               "tag": "latest",
               "repository": "group-a/project-a"
             }
           },
       {
             "action": "delete",
             "target": {
               "tag": "latest",
               "repository": "group-a/project-a"
             }
           }
         ]
     }'
  6. Refresh the sidekiq dashboard - you should see one scheduled job for UpdateProjectStatisticsWorker 🎉
  7. Re-hit the API endpoint multiple times and confirm only the one worker is enqueued in the step above - this confirms that multiple events in quick succession will only trigger the one project statistics update 👍🏽
  8. Optional: Once the scheduled job has been processed, you can now inspect your project_statistics record for your project to confirm the container_registry_size has updated to the value you specified in step 1 (out of scope for this MR, the update itself was not added in this MR)
  9. Optional: once the job queue is cleared, you could modify the events being sent to the API endpoint and confirm that a new job is not enqueued if the action is anything other than push or delete, or if the target.tag key is missing

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Vijay Hawoldar

Merge request reports