Skip to content

git: Increase locking timeout for loose references

When trying to update references, Git will first try to acquire a lock and write the new value to this lockfile. As those locks may also be acquired by concurrently running processes it uses a timeout during which it will spin around the lock until it finally gets released. If Git fails to acquire the lock in time it aborts the transaction and returns an error.

We have recently bumped the timeout for acquiring the packed-refs lock to 1 second given that it is a shared resource that needs to be locked whenever deleting any reference. This new timeout value does not apply to loose references though, and that is on purpose: loose references only require fine-grained locking and thus don't have the same amount of lock contention as the packed-refs file.

But we have still observed locking issues here in production systems. During a storm of incoming RPC calls the system was heavily loaded. While there was some amount of contention around loose references, it still was comparatively limited. But given that:

- The system was heavily loaded and thus slowed down.

- The default loose reference locking timeout is only a 100
  milliseconds.

We saw thousands of WriteRef() RPC calls failing because of concurrently held locks.

Let's thus increase the timeout for loose references, as well. Given that the contention here should be much lower we don't go all the way to 10 seconds though, but instead only bump it to 1 second, which is the same 10-fold factor we have applied to the packed-refs locking timeout.

Changelog: fixed

Merge request reports