DeleteUserWorker has long running transactions
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
We recently introduced capturing database-transaction length as a metric in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29928. Now with this available in Prometheus, we discovered that DeleteUserWorker has potential to run a long database transaction, which we should investigate further.
From Prometheus, we can see that within last 12 hours, there were about 150 transaction for said job that took longer than 10s.
The impact from that is that this job would hold a backend connection to the database until it commits the transaction. This can lead to starvation of resources and delay other jobs.
Now the frequency is not very high of said job. Still creating this issue here as we're starting to investigate these type of problems and this may serve as a good example.
cc @andrewn