N+1 query building keep around refs in DiffNote
While running a simple import from a Bitbucket project, this halted my import:
15:21:52 rails-background-jobs.1 | 2018-06-28T22:21:52.275Z 57726 TID-owvu0z1iq WARN: Gitlab::GitalyClient::TooManyInvocationsError: GitalyClient#migrate called 72 times from single request. Potential n+1?
15:21:52 rails-background-jobs.1 | The following call site called into Gitaly 5 times:
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/gitaly_client.rb:274:in `migrate'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/git/repository.rb:1274:in `gitaly_migrate'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/git/repository.rb:265:in `ref_exists?'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/app/models/repository.rb:236:in `ref_exists?'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/metrics/instrumentation.rb:159:in `block in ref_exists?'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/metrics/method_call.rb:36:in `measure'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/metrics/instrumentation.rb:159:in `ref_exists?'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/app/models/repository.rb:257:in `kept_around?'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/metrics/instrumentation.rb:159:in `block in kept_around?'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/metrics/method_call.rb:36:in `measure'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/metrics/instrumentation.rb:159:in `kept_around?'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/app/models/repository.rb:248:in `keep_around'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/metrics/instrumentation.rb:159:in `block in keep_around'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/metrics/method_call.rb:36:in `measure'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/lib/gitlab/metrics/instrumentation.rb:159:in `keep_around'
15:21:52 rails-background-jobs.1 | /Users/stanhu/gdk/gitlab/app/models/diff_note.rb:192:in `keep_around_commits'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:432:in `block in make_lambda'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:228:in `block in halting_and_conditional'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:506:in `block in call'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:506:in `each'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:506:in `call'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/callbacks.rb:302:in `create_or_update'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/persistence.rb:142:in `save!'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/validations.rb:43:in `save!'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/attribute_methods/dirty.rb:29:in `save!'
15:21:52 rails-background-jobs.1 | /Users/stanhu/.rbenv/versions/2.4.4/lib/ruby/gems/2.4.0/gems/activerecord-4.2.10/lib/active_record/transactions.rb:291:in `block in save!'
I assume this is because Repository#keep_around
does a number calls? I assume it's not because DiffNote
creates multiple keep around refs, although that can't help either.
def keep_around(sha)
return unless sha.present? && commit_by(oid: sha)
return if kept_around?(sha)
# This will still fail if the file is corrupted (e.g. 0 bytes)
raw_repository.write_ref(keep_around_ref_name(sha), sha, shell: false) # 1 here
rescue Gitlab::Git::CommandError => ex
Rails.logger.error "Unable to create keep-around reference for repository #{path}: #{ex}"
end
It sounds like we just this move this entire function into Gitaly?
Edited by Stan Hu