After 18.4 all gitaly.CommitService/CommitLanguages requests fail transactions with no-votes causing constant replication
We're running a self-hosted GitLab Premium in K8s. Immediately after we updated to 18.4 two days ago we began seeing a huge increase in no-votes
and outdated
tx result values reported through the gitaly_praefect_tx_replications_total
metric and "transactional node states" log message in Praefect. We have been able to narrow down these issues to be occurring only with gitaly.CommitService/CommitLanguages
and gitaly.RepositoryService/FetchRemote
. With gitaly.RepositoryService/FetchRemote
it seems like we were having them even before the update, and they're rare, so we didn't see any impact from them. With gitaly.CommitService/CommitLanguages
, however, they occur on literally every commit to any repo (each commit results in a gitaly.CommitService/CommitLanguages
call from Gitaly which then fails with no-votes
and causes the replicas to be resynced).