Perform failovers lazily

Praefect's PerRepositoryElector runs elections globally when Praefect launches and when a Gitaly node's health status changed. This approach was originally taken to match global elections done by the sqlElector as well. While the sqlElector runs elections after every health check, by default every 3s, the event driven approach was implemented for the PerRepositoryElector as it has to perform a lot more work every election run compared to the sqlElector. The sqlElector has a single primary for each virtual storage where as the PerRepositoryElector has a primary record for every repository. While both electors check every repository's generations to pick the best new primary, only the PerRepositoryElector has to write potentially a large number of records as well. We can do a lot better though:

1. If the primary is unavailable only temporarily, there's a high chance that the repository is not even accesed during the outage. If so, there's no need to eagerly failover as no one would even see the failure.

2. Most of the operations on the repositories are reads. Reads can be served from any up to date replica without needing to have a primary. Only once an RPC that requires the primary arrives we care about having a healthy primary.

Given the above, this commit implements a lazy approach to failovers. This removes the background election loop entirely and elects a primary if needed when an RPC requires a primary. This happens transparently when getting the primary from the database. This brings multiple benefits:

1. Perfomance improves as we don't have to perform failovers for repositories which are not written to during the primary's outage. This reduces the time to perfrom failovers as we are working on records of a single repository as opposed to all of the repositories.

2. Failover code is responsive without having to feed it more and more events. This becomes more relevant as we implement rebalancing features. When moving a repository with a single replica, we may have to demote the primary temporarily and we want it to be re-elected as soon as a request needs it and it's possible. Previous approach would require us hooking more code into the events where as this lazy approach just works.

3. It's easier to reason about synchronous code rather than asynchronous elections.

4. We can log all the individual changes, as opposed to logging the aggregate stats of demotions and promotions.

Closes: #3207 (closed)
Documentation: gitlab!62704 (merged)

This is marked as a draft until both !3549 (merged) and !3548 (merged) are ready to go. This and the two MRs should be merged at the same time so the reporting tools match the failover behavior.