Gracefully handle stale lock files in git repositories

In https://gitlab.com/gitlab-com/infrastructure/issues/2893, we see increased incidents of stale git lock files lingering around preventing pushes and updates.

Other relevant issues:

@pcarranza asks:

this seems to be popping up a lot of times, could we investigating having a way to manage this from the application itself? Probably something on the "housekeeping" panel could make the repo read only and perform a clean operation on it.

This is tricky. Lock files exist for a reason. I wish we could reproduce the issue in some way. I'd hesitate to have a blanket Housekeeping button that wipes lock files there. It should only purge them if the files exist for longer than some time.

@DouweM @andrewn @smcgivern Thoughts on how best to handle this?

Edited by Stan Hu