Traffic not always directed to Canary even though `gitlab_canary=true` cookie is set
Summary
This is an intermittent issue. Traffic is not always directed to Canary even though gitlab_canary=true
cookie is set.
This was discovered by e2e tests run against production canary.
- One of the failure issue: #431759 (closed)
- Pipeline: https://ops.gitlab.net/gitlab-org/quality/canary/-/pipelines/2523337
- Slack: https://gitlab.slack.com/archives/CCNNKFP8B/p1699980431008749
Background:
There was an incident where e2e tests were failing against gstg-cny and it was doubted that although the e2e tests were expecting to hit canary
, they were actually hitting main
. To confirm this theory, a validation was added here: !135937 (merged)
Steps to reproduce
There are no reproducible steps as this happens once in a while
Example Project
What is the current bug behavior?
Traffic is sometimes directed to main
even though gitlab_canary=true
cookie is set.
What is the expected correct behavior?
Traffic should always be directed to canary
when gitlab_canary=true
cookie is set.
Relevant logs and/or screenshots
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)