Skip to content

Fix multiple calls to delete refs in Deployments::ArchiveInProjectWorker sidekiq worker

Problem

While researching issues affecting the release stage error budget, an issue was detected in Deployments::ArchiveInProjectWorker.

This happens because multiple calls to delete refs via Gitaly are being executed, which under the hood seem to result in a git error while trying lock all references in preparation for deleting a specific reference, as can be seen in the following error:

unable to delete refs: state update to "prepare" failed: EOF, stderr: "fatal: prepare: multiple updates for 'test/test_test' (including one via symref 'refs/environments/test15/deployments/692189') are not allowed\n"

This error affects the sidekiq_execution component of error budget. Check links below for further information.

Logs: https://log.gprd.gitlab.net/goto/6e965c20-4a45-11ed-8d37-e9a2f393ea2a

Sentry: https://sentry.gitlab.net/gitlab/gitlabcom/issues/3308826

Proposal

Update ArchiveInProjectService to pass --no-deref option in the gitaly call to delete refs.

Note: The --no-deref option was implemented in gitaly#4624 (closed).

Edited by Ahmed Hemdan