Geo repository sync worker attempts to sync repos on unhealthy shards in non-backfill conditions
Summary
As seen on the Geo testbed. We filter by healthy shards for backfill, but not when picking up syncs that are coming from the Geo log cursor.
Steps to reproduce
Send a RepositoryUpdatedEvent on the primary, for a project whose shard is unhealthy
What is the current bug behavior?
The secondary tries and fail to update that project. Repeatedly.
What is the expected correct behavior?
The initial attempt should be permitted. When that has failed, the failing entry should not be tried again until its shard is healthy.
Possible fixes
Edited by Nick Thomas