Geo: Repository sync appears to be retrying the same projects
We got Geo able to clone in a test setup across Azure and GCE now. The admin page shows:
300 failed projects seems low considering most of the projects should fail right away because only one NFS shard is available.
If you use
jq to parse the data, you can see that there were about 24,000 attempts to run
grep RepositorySync geo.log | jq '.project_id' | grep -v null | wc -l 24020
But there were only 300 unique project IDs attempted:
cat geo.log | grep RepositorySync | jq '.project_id' | sort | uniq | wc -l 302
It seems that we are trying the same projects over without consideration that the last repository failed.
(confidential logs removed)