Redis cache inconsistency between canary and production
On web-cny-01-sv-gprd, I am seeing odd behavior where the cache does not seem to be consistent between canary and production. I noticed this because my source branch in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23607 claims not to exist:
On web-cny-02:
irb(main):018:0> proj = Project.find(13083)
=> #<Project id:13083 gitlab-org/gitlab-ce>
irb(main):019:0> proj.repository.branch_names.include?('sh-remove-gitlab-shell-include')
=> false
irb(main):020:0> proj.repository.branch_names.count
=> 2226
On web-01-sv-grpd:
irb(main):011:0> proj = Project.find(13083)
=> #<Project id:13083 gitlab-org/gitlab-ce>
irb(main):012:0> proj.repository.branch_names.include?('sh-remove-gitlab-shell-include')
=> true
irb(main):013:0> proj.repository.branch_names.count
=> 2230
I checked that both nodes are using the same /var/opt/gitlab/gitlab-rails/etc/redis.cache.yml
, although it's possible the application on canary is ignoring that file or using a different cache key due to the Rails 5 upgrade.
Edited by Stan Hu