Improve observability of project mirroring

As it is now, we are mostly blind to the state of project mirroring. One of the consequences of that is that we are not alerting on project mirroring delay.

Current state

The state of mirroring is stored in project_mirror_data table in Postgres. gitlab_exporter is surfacing this information using a number of SQL queries: https://gitlab.com/gitlab-org/gitlab-exporter/-/blob/master/lib/gitlab_exporter/database/row_count.rb#L47 . The metrics from gitlab_exporter are visualized on this dashboard: https://dashboards.gitlab.net/d/_MKRXrSmk/pull-mirrors?orgId=1&from=now-3h&to=now

Idea

One way to improve this situation is to have the sidekiq workers report the relevant metrics since they have all the information necessary to calculate them. This would also keep the instrumentation next to the implementation.

Related incidents