ee:geo QA specs are failing as of https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/7210
https://gitlab.com/gitlab-org/gitlab-qa/-/jobs/94212855
Failures:
1) geo Geo Nodes API Geo Nodes API on primary node GET /geo_nodes/status
Failure/Error: expect(json_body.size).to be >= 2
expected: >= 2
got: 1
Shared Example Group: "retrieving status about all Geo nodes" called from ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:91
# ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:35:in `block (4 levels) in <module:QA>'
# ./qa/specs/runner.rb:28:in `perform'
# ./qa/scenario/template.rb:8:in `block in perform'
# ./qa/scenario/template.rb:6:in `tap'
# ./qa/scenario/template.rb:6:in `perform'
# ./qa/ee/scenario/test/geo.rb:33:in `perform'
# ./qa/scenario/template.rb:8:in `block in perform'
# ./qa/scenario/template.rb:6:in `tap'
# ./qa/scenario/template.rb:6:in `perform'
# ./qa/scenario/bootable.rb:26:in `launch!'
2) geo Geo Nodes API Geo Nodes API on primary node GET /geo_nodes/:id/status of secondary node
Failure/Error: expect_status(200)
expected: 200
got: 404
(compared using ==)
Shared Example Group: "retrieving status about a specific Geo node" called from ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:92
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:36:in `expect_status'
# ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:62:in `block (4 levels) in <module:QA>'
# ./qa/specs/runner.rb:28:in `perform'
# ./qa/scenario/template.rb:8:in `block in perform'
# ./qa/scenario/template.rb:6:in `tap'
# ./qa/scenario/template.rb:6:in `perform'
# ./qa/ee/scenario/test/geo.rb:33:in `perform'
# ./qa/scenario/template.rb:8:in `block in perform'
# ./qa/scenario/template.rb:6:in `tap'
# ./qa/scenario/template.rb:6:in `perform'
# ./qa/scenario/bootable.rb:26:in `launch!'
3) geo Geo Nodes API Geo Nodes API on primary node repairing a Geo node POST /geo_nodes/:id/repair for secondary node
Failure/Error: expect_json(geo_node_id: @secondary_node[:id])
Airborne::ExpectationError:
Expected NilClass
to be an object with property geo_node_id
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:165:in `rescue in extract_actual'
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:161:in `extract_actual'
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:89:in `block in expect_json_impl'
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:87:in `each'
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:87:in `expect_json_impl'
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:19:in `block in expect_json'
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:139:in `call_with_path'
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:18:in `expect_json'
# ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:139:in `block (5 levels) in <module:QA>'
# ./qa/specs/runner.rb:28:in `perform'
# ./qa/scenario/template.rb:8:in `block in perform'
# ./qa/scenario/template.rb:6:in `tap'
# ./qa/scenario/template.rb:6:in `perform'
# ./qa/ee/scenario/test/geo.rb:33:in `perform'
# ./qa/scenario/template.rb:8:in `block in perform'
# ./qa/scenario/template.rb:6:in `tap'
# ./qa/scenario/template.rb:6:in `perform'
# ./qa/scenario/bootable.rb:26:in `launch!'
# ------------------
# --- Caused by: ---
# NoMethodError:
# undefined method `[]' for nil:NilClass
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:163:in `extract_actual'
4) geo Geo Nodes API Geo Nodes API on secondary node GET /geo_nodes/:id/status of secondary node
Failure/Error: expect_status(200)
expected: 200
got: 404
(compared using ==)
Shared Example Group: "retrieving status about a specific Geo node" called from ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:160
# /usr/local/bundle/gems/airborne-0.2.13/lib/airborne/request_expectations.rb:36:in `expect_status'
# ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:62:in `block (4 levels) in <module:QA>'
# ./qa/specs/runner.rb:28:in `perform'
# ./qa/scenario/template.rb:8:in `block in perform'
# ./qa/scenario/template.rb:6:in `tap'
# ./qa/scenario/template.rb:6:in `perform'
# ./qa/ee/scenario/test/geo.rb:33:in `perform'
# ./qa/scenario/template.rb:8:in `block in perform'
# ./qa/scenario/template.rb:6:in `tap'
# ./qa/scenario/template.rb:6:in `perform'
# ./qa/scenario/bootable.rb:26:in `launch!'
5) geo Geo Nodes API Geo Nodes API on secondary node GET /geo_nodes/status
Failure/Error: expect(json_body.size).to be >= 2
expected: >= 2
got: 1
Shared Example Group: "retrieving status about all Geo nodes" called from ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:159
# ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:35:in `block (4 levels) in <module:QA>'
# ./qa/specs/runner.rb:28:in `perform'
# ./qa/scenario/template.rb:8:in `block in perform'
# ./qa/scenario/template.rb:6:in `tap'
# ./qa/scenario/template.rb:6:in `perform'
# ./qa/ee/scenario/test/geo.rb:33:in `perform'
# ./qa/scenario/template.rb:8:in `block in perform'
# ./qa/scenario/template.rb:6:in `tap'
# ./qa/scenario/template.rb:6:in `perform'
# ./qa/scenario/bootable.rb:26:in `launch!'
Top 10 slowest examples (178.53 seconds, 96.9% of total time):
geo GitLab Geo repository replication users pushes code to the primary node
110.28 seconds ./qa/specs/features/browser_ui/ee_geo/repository_replication_spec.rb:6
geo GitLab Geo project rename replication user renames project
35.99 seconds ./qa/specs/features/browser_ui/ee_geo/rename_replication_spec.rb:6
geo GitLab Geo attachment replication user uploads attachment to the primary node
30.67 seconds ./qa/specs/features/browser_ui/ee_geo/attachment_replication_spec.rb:8
geo Geo Nodes API Geo Nodes API on secondary node GET /geo_nodes/status
0.57035 seconds ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:31
geo Geo Nodes API Geo Nodes API on secondary node GET /geo_nodes is not current
0.29776 seconds ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:163
geo Geo Nodes API Geo Nodes API on secondary node GET /geo_nodes/current/failures
0.27203 seconds ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:74
/builds/gitlab-org/gitlab-qa/lib/gitlab/qa/docker/shellout.rb:31:in `block in execute!': Docker command `docker run -t --rm --net=geo --env GITHUB_ACCESS_TOKEN="$GITHUB_ACCESS_TOKEN" --env EE_LICENSE="$EE_LICENSE" --env GCLOUD_ACCOUNT_EMAIL="$GCLOUD_ACCOUNT_EMAIL" --env GCLOUD_ACCOUNT_KEY="$GCLOUD_ACCOUNT_KEY" --env CLOUDSDK_CORE_PROJECT="$CLOUDSDK_CORE_PROJECT" --env GCLOUD_ZONE="$GCLOUD_ZONE" --volume /var/run/docker.sock:/var/run/docker.sock:z --volume /builds/gitlab-org/gitlab-qa/gitlab-qa-run-2018-09-03-14-04-06-2febcc90/gitlab-ee-qa-f6ff17e5:/home/qa/tmp:z --name gitlab-ee-qa-f6ff17e5 registry.gitlab.com/gitlab-org/omnibus-gitlab/gitlab-ee-qa:8f498beac241153b404362f24cfe3dcfa95f9bd5 QA::EE::Scenario::Test::Geo --primary-address http://gitlab-primary.geo --primary-name gitlab-primary --secondary-address http://gitlab-secondary.geo --secondary-name gitlab-secondary` failed! (Gitlab::QA::Docker::Shellout::StatusError)
from /usr/local/lib/ruby/2.4.0/open3.rb:205:in `popen_run'
from /usr/local/lib/ruby/2.4.0/open3.rb:190:in `popen2e'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/docker/shellout.rb:19:in `execute!'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/docker/command.rb:36:in `execute!'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/docker/engine.rb:22:in `block in run'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/docker/engine.rb:16:in `tap'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/docker/engine.rb:16:in `run'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/component/specs.rb:24:in `perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/template.rb:8:in `block in perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/template.rb:6:in `tap'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/template.rb:6:in `perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/test/integration/geo.rb:58:in `block (4 levels) in perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/actable.rb:6:in `instance_exec'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/actable.rb:6:in `act'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/test/integration/geo.rb:48:in `block (3 levels) in perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/actable.rb:15:in `perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/test/integration/geo.rb:36:in `block (2 levels) in perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/component/gitlab.rb:60:in `instance'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/test/integration/geo.rb:35:in `block in perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/actable.rb:15:in `perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/test/integration/geo.rb:19:in `perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/template.rb:8:in `block in perform'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/template.rb:6:in `tap'
from /builds/gitlab-org/gitlab-qa/lib/gitlab/qa/scenario/template.rb:6:in `perform'
from bin/qa:8:in `<main>'
geo Geo Nodes API Geo Nodes API on primary node GET /geo_nodes/status
0.13845 seconds ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:31
geo Geo Nodes API Geo Nodes API on primary node editing a Geo node PUT /geo_nodes/:id for secondary node
0.13737 seconds ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:102
geo Geo Nodes API Geo Nodes API on secondary node GET /geo_nodes/:id/status of secondary node
0.09877 seconds ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:59
geo Geo Nodes API Geo Nodes API on primary node GET /geo_nodes/:id/status of primary node
0.07615 seconds ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:52
Top 4 slowest example groups:
geo
110.29 seconds average (110.29 seconds / 1 example) ./qa/specs/features/browser_ui/ee_geo/repository_replication_spec.rb:4
geo
35.99 seconds average (35.99 seconds / 1 example) ./qa/specs/features/browser_ui/ee_geo/rename_replication_spec.rb:4
geo
30.68 seconds average (30.68 seconds / 1 example) ./qa/specs/features/browser_ui/ee_geo/attachment_replication_spec.rb:4
geo
0.28365 seconds average (7.37 seconds / 26 examples) ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:4
Finished in 3 minutes 4.3 seconds (files took 3 minutes 26.1 seconds to load)
29 examples, 5 failures
Failed examples:
rspec './qa/specs/features/api/ee_geo/geo_nodes_spec.rb[1:1:1:3]' # geo Geo Nodes API Geo Nodes API on primary node GET /geo_nodes/status
rspec './qa/specs/features/api/ee_geo/geo_nodes_spec.rb[1:1:1:5]' # geo Geo Nodes API Geo Nodes API on primary node GET /geo_nodes/:id/status of secondary node
rspec ./qa/specs/features/api/ee_geo/geo_nodes_spec.rb:135 # geo Geo Nodes API Geo Nodes API on primary node repairing a Geo node POST /geo_nodes/:id/repair for secondary node
rspec './qa/specs/features/api/ee_geo/geo_nodes_spec.rb[1:1:2:5]' # geo Geo Nodes API Geo Nodes API on secondary node GET /geo_nodes/:id/status of secondary node
rspec './qa/specs/features/api/ee_geo/geo_nodes_spec.rb[1:1:2:3]' # geo Geo Nodes API Geo Nodes API on secondary node GET /geo_nodes/statusIt wasn't failing before, as far as I can see.