`RepositoryShardSyncWorker` does not seem to be running
I’m not sure if RepositoryShardSyncWorker
is able to run, as I get this error when getting healthy shard count:
[ gprd ] production> Gitlab::Geo::ShardHealthCache.healthy_shard_count
Redis::CannotConnectError: No sentinels available.
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:567:in `sentinel_detect'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:571:in `resolve_master'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:535:in `resolve'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:331:in `establish_connection'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:101:in `block in connect'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:293:in `with_reconnect'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:100:in `connect'
from /opt/gitlab/embedded/service/gitlab-rails/config/initializers/8_metrics.rb:217:in `connect'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:364:in `ensure_connected'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:221:in `block in process'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:306:in `logging'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:220:in `process'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis/client.rb:120:in `call'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/peek-redis-1.2.0/lib/peek/views/redis.rb:9:in `call'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis.rb:1291:in `block in scard'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/redis-3.3.5/lib/redis.rb:58:in `block in synchronize'
... 4 levels...
from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:17:in `block in with'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.1/lib/connection_pool.rb:64:in `block (2 levels) in with'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.1/lib/connection_pool.rb:63:in `handle_interrupt'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.1/lib/connection_pool.rb:63:in `block in with'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.1/lib/connection_pool.rb:60:in `handle_interrupt'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/connection_pool-2.2.1/lib/connection_pool.rb:60:in `with'
from /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/redis/wrapper.rb:17:in `with'
from /opt/gitlab/embedded/service/gitlab-rails/ee/lib/gitlab/geo/shard_health_cache.rb:39:in `healthy_shard_count'
from (irb):19
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.10/lib/rails/commands/console.rb:110:in `start'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.10/lib/rails/commands/console.rb:9:in `start'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.10/lib/rails/commands/commands_tasks.rb:68:in `console'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.10/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.10/lib/rails/commands.rb:17:in `<top (required)>'
from bin/rails:14:in `require'
from bin/rails:14:in `<main>'
Also it looks like RepositoryShardSyncWorker
is not running for the last hour or so since the GPRD repo sync failures were stopped here #282 (comment 68706985).