Define SLAs for Mirroring feature

Quoting @pcarranza comment on https://gitlab.com/gitlab-org/gitlab-ee/issues/3608#note_42392626

Ok, quick notes:

• Define an SLA:

  • External SLA (customer facing): no project mirror should go over 5hs since the last execution without being serviced.

  • Internal SLA (service health): no project mirror should spend more than 10m in between it being scheduled and it being serviced.

• Actions:

  • Stop talking about import projects to focus on mirrors only, reduce the noise.

  • Create queries to measure the SLIs for the proposed SLAs using the prometheus postgres exporter arbitrary queries configuration (https://gitlab.com/gitlab-cookbooks/gitlab-prometheus/blob/master/files/default/postgres_exporter/queries.yaml)

Once we have the SLIs in place, we will check if we are meeting the SLAs or not, and then continue the conversation there.

Tasklist

• gitlab-cookbooks/gitlab-prometheus!320 (merged) - Add External SLA: no project mirror should go over 5hs since the last execution without being serviced.
• gitlab-cookbooks/gitlab-prometheus!319 (merged) - Add Internal SLA: after setting next_execution_time no project should take more than next_execution_time + 10m to be serviced.

NOTE: I changed the Internal SLA because we need to make the distinction of setting next_execution_time and switching state to scheduled to avoid any confusion in the future.

cc/ @tiagonbotelho @DouweM