Skip to content

Garbage collector runs after each pull mirror schedule

Summary

Garbage collection runs after a repo with pull mirroring runs the mirror sync no matter it has to gc or not according to the commit counter.

This garbage collection runs on projects that were imported from Github (or other sources) and have pull mirroring configured.

This gc causes a big impact on storage if they are big enough. Maybe something changed on the way the Garbage collection is done from v11.8 and v11.9?

Steps to reproduce

  • Configure a project with pull mirroring
  • the project needs to have import_type="github" or any other type included in Gitlab::ImportSources.importer_names:
    • ["github", "bitbucket", "bitbucket_server", "gitlab", "google_code", "fogbugz", "gitlab_project", "gitea"]

Example Project

Cannot reproduce on gitlab.com

Relevant logs and/or screenshots

On https://gitlab.com/gitlab-org/gitlab-ee/blob/v11.9.9-ee/app/models/project_import_state.rb#L45-58 this check runs Projects::AfterImportService.new(project).execute after a pull mirror sync when import_type has one of the available values.

All our projects with pull mirroring from github have nil on this field, except one.

Results of GitLab environment info

Expand for output related to GitLab environment info
System information
System:
Proxy:          no
Current User:   git
Using RVM:      no
Ruby Version:   2.5.3p105
Gem Version:    2.7.6
Bundler Version:1.16.6
Rake Version:   12.3.2
Redis Version:  3.2.12
Git Version:    2.18.1
Sidekiq Version:5.2.5
Go Version:     unknown

GitLab information Version: 11.9.9-ee Revision: 0ee466e Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql DB Version: 9.6.2 URL: https://gitlab.cern.ch HTTP Clone URL: https://gitlab.cern.ch/some-group/some-project.git SSH Clone URL: ssh://git@gitlab.cern.ch:7999/some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: yes Using Omniauth: yes Omniauth Providers: saml, kerberos_spnego

GitLab Shell Version: 8.7.1 Repository storage paths:

  • default: /var/opt/gitlab/git-data/repositories
  • secondary: /var/opt/gitlab/git-data2/repositories
  • third: /var/opt/gitlab/git-data3/repositories
  • fourth: /var/opt/gitlab/git-data4/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git
Edited by Daniel Juarez