Skip to content

Ensuring all gitaly nodes are running prior to e2e

John McDonnell requested to merge jmd-fix-gitaly-distributed-reads-e2e into master

Description of the test

Fixes #344240 (closed) It appears that when de-quarantining some other tests, we may have left the containers in a state where some nodes were not in a 'started' state when this test began to run. Hardening this test by ensuring we always start all gitaly nodes before the test so we aren't dependant on other tests leaving the environment in the correct state.

I've observed the following error in the failures for this issue. Error while dialing dial tcp: lookup gitaly1.test on 127.0.0.11:53: no such host The failure to lookup the hostname on port 53 indicates some type of DNS issue within the docker containers, which I've only been able to easily reproduce when a container is 'stopped' but as the test is flaky, it's hard to pin down exactly.

Check-list

  • Confirm the test has a testcase: tag linking to an existing test case in the test case project.
  • Note if the test is intended to run in specific scenarios. If a scenario is new, add a link to the MR that adds the new scenario.
  • Follow the end-to-end tests style guide and best practices.
  • Use the appropriate RSpec metadata tag(s).
  • Ensure that a created resource is removed after test execution. A Group resource can be shared between multiple tests. Do not remove it unless it has a unique path. Note that we have a cleanup job that periodically removes groups under gitlab-qa-sandbox-group.
  • Ensure that no transient bugs are hidden accidentally due to the usage of waits and reloads.
  • Verify the tags to ensure it runs on the desired test environments.
  • If this MR has a dependency on another MR, such as a GitLab QA MR, specify the order in which the MRs should be merged.
  • (If applicable) Create a follow-up issue to document the special setup necessary to run the test: ISSUE_LINK
  • If the test requires an admin's personal access token, ensure that the test passes on your local with and without the GITLAB_QA_ADMIN_ACCESS_TOKEN provided.
Edited by John McDonnell

Merge request reports