This was brought up on the 2018-03-06 on-call handover.
Might be related to this latency increase on the api:
https://performance.gitlab.net/dashboard/db/rails-latencies?refresh=5m&panelId=11&fullscreen&orgId=1&from=now-30d&to=now
It does look like we are running a bit hot on the fleet, we may want to look into adding capacity in the meantime. /cc @gl-infra