Stop creating keep-around refs
Release notes
Problem to solve
Keep-around refs are created to reference commits that would otherwise be garbage collected. This is done so that even when the user force-pushes, the original commit can still be referenced by gitlab comments/merge-requests/environments/pipelines. Keep-around refs are named after the commit object ID (SHA) and because of this it is difficult to determine what they were created for, and so difficult to ever delete. This means that a busy repository will quickly build up an enormous amount of refs affecting the performance of the git repository.
Keep-around refs are hidden from the user. There is no way for a user to explicitly update or remove keep-around refs. This means that accidental pushes (large binaries, secrets, etc) can be kept and are then difficult to clean up properly.
Without keep-around refs, unreachable objects get removed by housekeeping after 2-weeks unless an admin triggers housekeeping manually, then they can be removed after 30 minutes. See https://docs.gitlab.com/ee/administration/housekeeping.html#prune-unreachable-objects
Proposal
- Determine which keep-around refs are strictly required for the functionality of gitlab.
- Replace these keep-around refs with scoped refs (e.g
heads/merge-requests/<merge-request-id>/<version>) that have a determined life-cycle (that is there must be a condition under which it will or can be deleted). - Any keep-around refs that were determined not to be strictly required should no longer be created.
- Deprecate and remove all keep-around refs.
Intended users
Feature Usage Metrics
Current keep-around refs
Ci::Pipeline
app/models/ci/pipeline.rb:1394: project.repository.keep_around(self.sha, self.before_sha)| Which commit | When | Condition |
|---|---|---|
| The commit that the pipeline is running on. | after_create |
Unless importing and if associated with project |
| The previous commit present on a branch or tag. | after_create |
Unless importing and if associated with project |
MergeRequest
app/models/merge_request.rb:1881: project.repository.keep_around(self.merge_commit_sha)
app/services/merge_requests/cleanup_refs_service.rb:70: repository.keep_around(ref_head_sha, merge_ref_sha)| Which commit | When | Condition |
|---|---|---|
| The merge commit. | after_save |
Unless importing. |
refs/merge_requests/<iid>/merge |
Merge or close. | |
refs/merge_requests/<iid>/head |
Merge or close. |
MergeRequestDiff
app/models/merge_request_diff.rb:827: repo.keep_around(start_commit_sha, head_commit_sha, base_commit_sha)
DiffNote
app/models/diff_note.rb:190: repository.keep_around(*shas)| Which commit | When | Condition |
|---|---|---|
| The target branch head. | after_create |
Unless importing |
Note
app/models/note.rb:818: project.repository.keep_around(self.commit_id)| Which commit | When | Condition |
|---|---|---|
| The commit that was commented on. | after_save |
When associated with a project (not a personal snippet), and unless importing or unless publishing drafts (as this is handled by DraftNote::PublishService). |
DraftNotes::PublishService
app/services/draft_notes/publish_service.rb:108: project.repository.keep_around(*shas)| Which commit | When | Condition |
|---|---|---|
| The commit that has a review comment on. | On review submit |
SentNotification
app/models/sent_notification.rb:128: project.repository.keep_around(self.commit_id)| Which commit | When | Condition |
|---|---|---|
| The commit associated with the notification. | On notification. |
Todo
app/models/todo.rb:297: project.repository.keep_around(self.commit_id)| Which commit | When | Condition |
|---|---|---|
| The commit associated with the todo. | after_save |