Geo: Failed to remove git config error
On a Web console, I attempted to run:
Geo::ProjectRegistry.dirty.find_each do |registry|
Geo::ProjectSyncWorker.new.perform(registry.project_id, Time.now)
end
Occasionally, I hit this error, causing this task to stop:
Rugged::OSError: failed to read descriptor: No such file or directory
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/models/ee/repository.rb:23:in `delete'
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/models/ee/repository.rb:23:in `block (2 levels) in with_config'
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/models/ee/repository.rb:23:in `each'
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/models/ee/repository.rb:23:in `block in with_config'
from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/gitaly_client/storage_settings.rb:29:in `block in allow_disk_access'
from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/temporarily_allow.rb:7:in `temporarily_allow'
from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/gitaly_client/storage_settings.rb:29:in `allow_disk_access'
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/models/ee/repository.rb:22:in `with_config'
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/services/geo/base_sync_service.rb:102:in `fetch_geo_mirror'
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/services/geo/base_sync_service.rb:56:in `fetch_repository'
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/services/geo/wiki_sync_service.rb:8:in `sync_repository'
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/services/geo/base_sync_service.rb:28:in `block in execute'
from /opt/gitlab/embedded/service/gitlab-rails/app/services/concerns/exclusive_lease_guard.rb:22:in `try_obtain_lease'
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/services/geo/base_sync_service.rb:25:in `execute'
from /opt/gitlab/embedded/service/gitlab-rails/ee/app/workers/geo/project_sync_worker.rb:33:in `perform'
from (irb):233:in `block in irb_binding'
from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:51:in `block (2 levels) in find_each'
from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:51:in `each'
from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:51:in `block in find_each'
from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:124:in `find_in_batches'
from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:50:in `find_each'
from (irb):232
from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/commands/console.rb:110:in `start'
from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/commands/console.rb:9:in `start'
from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/commands/commands_tasks.rb:68:in `console'
from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
from /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/commands.rb:17:in `<top (required)>'
from bin/rails:14:in `require'
from bin/rails:14:in `<main>'
This seems like it's happening because another job does a sync and removes the existing git config file. However, there is an exclusive lease surrounding the project ID, so I wouldn't expect this to happen.