Remove Ci::PersistentRef in favour of bare SHA checkout

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Release notes

Problem to solve

The Ci::PersistentRef logic exists so that the runner always has a fetchable ref (see original issue). But it is:

• Fallible: There is a race between ref deletion and ref creation. When a pipeline completes, we delete the ref. But completion is not permanent. Manual job runs can move the pipeline back into a running status. And when a build starts, we try to resurrect it. The order of those events is not guaranteed, meaning that we may accidentally delete a ref that is still needed.
• Redundant: Git can fetch objects directly. These objects won't be garbage collected, because we create a keep-around ref for pipeline.sha. The persistent ref life cycle also creates unnecessary churn on Gitaly. There is a tremendous amount of refs being created and deleted all the time that we shouldn't need at all.

Proposal

Deprecate and phase out the use of Ci::PersistentRef in favour of fetching the SHA directly.

Intended users

Feature Usage Metrics

Does this feature require an audit event?