Workspaces setup done in runtime in CI
All threads resolved!
All threads resolved!
Compare changes
Files
3+ 50
− 0
This MR is the implementation of the Support for Workspaces to be setup in the runtime in the CI and also support using existing cluster - Refer #397005 (closed) and #414915 (closed)
The existing E2E test is moved under qa/qa/specs/features/ee/browser_ui/3_create/remote_development/without_setup/workspace_actions_without_setup_spec.rb
This test is in Quarantine and it is still used by the developers for local testing. This test may be removed in the future when its not needed. It test can be run using the - scripts/remote_development/run-e2e-tests.sh
Pre-requisites for this E2E test
This E2E tests will do the following:
This test can be run by
DEVFILE_PROJECT="devfile-test-project" AGENT_NAME="test-agent" bundle exec bin/qa Test::Instance::All "$TEST_INSTANCE_URL" -- --tag quarantine qa/specs/features/ee/browser_ui/3_create/remote_development/without_setup
The new E2E test is under qa/qa/specs/features/ee/browser_ui/3_create/remote_development/workspace_actions_spec.rb
This is the implementation to install and then create workspace from scratch
Pre-requisites for this E2E test
This E2E tests will do the following:
If WORKSPACES_CLUSTER_AVAILABLE=false (default case)
Finally, Once the test is complete the cluster will be deleted, the agent and agent_token will be removed.
If WORKSPACES_CLUSTER_AVAILABLE=true
The next steps are same as above:
Finally, Once the test is complete the cluster will remain and only the agent and agent_token will be removed.
New CI variables introduced in this change:
The MR to add these into terraform - https://gitlab.com/gitlab-org/quality/engineering-productivity-infrastructure/-/merge_requests/436
The below variables are not provisioned right now, and can be done once the cluster for the tests are in place
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
nivethaprabakaran@Nivetha qa % bundle exec bin/qa Test::Instance::All https://staging.gitlab.com -- -- qa/specs/features/ee/browser_ui/3_create/remote_development/workspace_actions_spec.rb
2023-08-01 18:14:56 WARN Selenium [:logger_info] Details on how to use and modify Selenium logger:
https://selenium.dev/documentation/webdriver/troubleshooting/logging#ruby
2023-08-01 18:14:56 WARN Selenium applicable driver not found; attempting to install with Selenium Manager
2023-08-01 18:14:58 / CONF ::
==> Base URL: https://staging.gitlab.com
==> Browser: #<Selenium::WebDriver::Chrome::Driver:0x000000010cd35768>
==> Libraries: Chemlab::Vendor
[Aug 01 2023 18:14:58 IST (QA Tests)] INFO -- Caching token for username: gitlab-qa-bot, last six chars of token:KSTdjm
[Aug 01 2023 18:14:58 IST (QA Tests)] INFO -- Caching token for username: gitlab-qa, last six chars of token:abiBtU
[Aug 01 2023 18:14:58 IST (QA Tests)] INFO -- Browser: chrome
[Aug 01 2023 18:14:58 IST (QA Tests)] INFO -- Performing sanity check for environment!
Run options: exclude {:orchestrated=>true, :transient=>true, :sanity_feature_flags=>true, :geo=>true, :skip_live_env=>true}
Randomized with seed 2590
Create
Remote Development
when setup is done in runtime
behaves like workspaces actions
[Aug 01 2023 18:15:01 IST (QA Tests)] INFO -- Starting test: Create Remote Development when setup is done in runtime behaves like workspaces actions creates a new workspace and then stops and terminates it
checking for kubectl... yes
checking for gcloud... yes
[Aug 01 2023 18:15:01 IST (QA Tests)] INFO -- Executing: `gcloud auth activate-service-account automated-qa-tests@group-qa-tests-566cc6.iam.gserviceaccount.com --key-file /var/folders/96/qv364xss6h75crxtybswsp1c0000gn/T/gcloud-account-key20230801-68626-djhnv2`
.
[Aug 01 2023 18:15:02 IST (QA Tests)] INFO -- Executing: `gcloud container clusters create qa-cluster-20230801124502-90451dab --region europe-west4 --disk-size 15GB --num-nodes 1 && gcloud container clusters get-credentials --region europe-west4 qa-cluster-20230801124502-90451dab `
............
[Aug 01 2023 18:21:25 IST (QA Tests)] INFO -- Executing: `curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 && chmod 700 get_helm.sh && DESIRED_VERSION=v3.7.0 ./get_helm.sh `
.
[Aug 01 2023 18:21:26 IST (QA Tests)] INFO -- ==> Retrieved a QA::Resource::Sandbox with full_path 'gitlab-qa-sandbox-group-3' via api in 0.6 seconds
[Aug 01 2023 18:21:28 IST (QA Tests)] INFO -- ==> Built a QA::Resource::Group with full_path 'gitlab-qa-sandbox-group-3/parent-group-to-test-remote-development-cbcce41978320fe4' via api in 2.54 seconds
[Aug 01 2023 18:21:30 IST (QA Tests)] INFO -- ==> Built a QA::Resource::Project with full_path 'gitlab-qa-sandbox-group-3/parent-group-to-test-remote-development-cbcce41978320fe4/agent-project-53d9fc1c6695ddbb' via api in 1.77 seconds
[Aug 01 2023 18:21:31 IST (QA Tests)] INFO -- ==> Built a QA::Resource::Clusters::Agent with name 'remotedev-5696df4e' via api in 0.49 seconds
[Aug 01 2023 18:21:31 IST (QA Tests)] INFO -- ==> Built a QA::Resource::Clusters::AgentToken with id '1024155' via api in 0.56 seconds
[Aug 01 2023 18:21:33 IST (QA Tests)] INFO -- ==> Built a QA::Resource::Repository::Commit via api in 1.73 seconds
[Aug 01 2023 18:21:35 IST (QA Tests)] INFO -- ==> Built a QA::Resource::Project with full_path 'gitlab-qa-sandbox-group-3/parent-group-to-test-remote-development-cbcce41978320fe4/devfile-project-e7c3718718279fb6' via api in 2.2 seconds
[Aug 01 2023 18:21:38 IST (QA Tests)] INFO -- ==> Built a QA::Resource::Repository::Commit via api in 2.26 seconds
[Aug 01 2023 18:21:38 IST (QA Tests)] INFO -- Executing: `helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx && helm repo update && helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx --create-namespace --version 4.3.0 `
.....................................................
[Aug 01 2023 18:22:26 IST (QA Tests)] INFO -- Executing: `helm repo add gitlab-workspaces-proxy https://gitlab.com/api/v4/projects/gitlab-org%2fremote-development%2fgitlab-workspaces-proxy/packages/helm/devel && helm repo update && helm upgrade --install gitlab-workspaces-proxy gitlab-workspaces-proxy/gitlab-workspaces-proxy --version 0.1.6 --namespace=gitlab-workspaces --create-namespace --set="auth.client_id=xxxxx" --set="auth.client_secret=xxxxx" --set="auth.host=https://staging.gitlab.com" --set="auth.redirect_uri=https://workspaces.staging.gitlabqa.dev/auth/callback" --set="auth.signing_key=123456789" --set="ingress.host.workspaceDomain=workspaces.staging.gitlabqa.dev" --set="ingress.host.wildcardDomain=*.workspaces.staging.gitlabqa.dev" --set="ingress.tls.workspaceDomainCert=xxxx" --set="ingress.tls.workspaceDomainKey=xxxx" --set="ingress.tls.wildcardDomainCert=xxxx" --set="ingress.tls.wildcardDomainKey=xxxx" --set="ingress.className=nginx" `
...............
[Aug 01 2023 18:22:40 IST (QA Tests)] INFO -- Executing: `helm repo add gitlab https://charts.gitlab.io && helm repo update && helm upgrade --install gitlab-agent gitlab/gitlab-agent --namespace "remotedev-5696df4e" --create-namespace --set image.tag=v16.0.0 --set config.token=xxxxxx --set config.kasAddress=wss://kas.staging.gitlab.com --set config.kasHeaders="{Cookie: gitlab_canary=false}" `
........................
[Aug 01 2023 18:22:50 IST (QA Tests)] INFO -- Executing: `kubectl -n ingress-nginx get svc ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}'`
[Aug 01 2023 18:22:53 IST (QA Tests)] INFO -- Executing: `gcloud dns record-sets update workspaces.staging.gitlabqa.dev --rrdatas= --ttl=300 --type=A --zone=gitlabqa-dev `
..
[Aug 01 2023 18:22:54 IST (QA Tests)] INFO -- Executing: `gcloud dns record-sets update "*.workspaces.staging.gitlabqa.dev" --rrdatas= --ttl=300 --type=A --zone=gitlabqa-dev `
..
[Aug 01 2023 18:22:55 IST (QA Tests)] INFO -- filling :login_field with "gitlab-qa"
[Aug 01 2023 18:22:55 IST (QA Tests)] INFO -- clicking the coordinates of :login_field
[Aug 01 2023 18:23:01 IST (QA Tests)] INFO -- filling :password_field with "*****"
[Aug 01 2023 18:23:01 IST (QA Tests)] INFO -- clicking the coordinates of :password_field
[Aug 01 2023 18:23:06 IST (QA Tests)] WARN -- Potentially Slow Code 'wait_for_requests ' took 0.578s
[Aug 01 2023 18:23:06 IST (QA Tests)] WARN -- Potentially Slow Code 'find_element password_field' took 0.594s
[Aug 01 2023 18:23:11 IST (QA Tests)] INFO -- clicking :sign_in_button
[Aug 01 2023 18:23:15 IST (QA Tests)] WARN -- Potentially Slow Code 'click_element sign_in_button' took 3.858s
[Aug 01 2023 18:23:18 IST (QA Tests)] INFO -- clicking :nav_item_link
[Aug 01 2023 18:23:19 IST (QA Tests)] WARN -- Potentially Slow Code 'click_element nav_item_link' took 1.216s
[Aug 01 2023 18:23:21 IST (QA Tests)] WARN -- Potentially Slow Code 'wait_for_requests ' took 1.851s
[Aug 01 2023 18:23:21 IST (QA Tests)] WARN -- Potentially Slow Code 'all_elements workspace_list_item' took 1.858s
[Aug 01 2023 18:23:21 IST (QA Tests)] INFO -- clicking :list_new_workspace_button
[Aug 01 2023 18:23:21 IST (QA Tests)] INFO -- clicking :workspace_devfile_project_id_field
[Aug 01 2023 18:23:23 IST (QA Tests)] WARN -- Potentially Slow Code 'wait_for_requests ' took 1.425s
[Aug 01 2023 18:23:23 IST (QA Tests)] WARN -- Potentially Slow Code 'click_element workspace_devfile_project_id_field' took 1.479s
[Aug 01 2023 18:23:23 IST (QA Tests)] INFO -- Searching in dropdown: "devfile-project-e7c3718718279fb6"
[Aug 01 2023 18:23:27 IST (QA Tests)] WARN -- Potentially Slow Code 'wait_for_requests ' took 1.998s
[Aug 01 2023 18:23:27 IST (QA Tests)] WARN -- Potentially Slow Code 'find_element workspace_cluster_agent_id_field' took 2.006s
[Aug 01 2023 18:23:27 IST (QA Tests)] INFO -- clicking :save_workspace_button
[Aug 01 2023 18:25:03 IST (QA Tests)] INFO -- clicking :workspace_stop_button
[Aug 01 2023 18:25:23 IST (QA Tests)] INFO -- clicking :workspace_terminate_button
[Aug 01 2023 18:25:43 IST (QA Tests)] INFO -- Executing: `gcloud container clusters delete --region europe-west4 qa-cluster-20230801124502-90451dab --quiet --async `
[Aug 01 2023 18:25:48 IST (QA Tests)] INFO -- Removed a QA::Resource::Clusters::AgentToken with id '1024155'
[Aug 01 2023 18:25:48 IST (QA Tests)] INFO -- Removed a QA::Resource::Clusters::Agent with name 'remotedev-5696df4e'
[Aug 01 2023 18:25:51 IST (QA Tests)] INFO -- Removed a QA::Resource::Group with full_path 'gitlab-qa-sandbox-group-3/parent-group-to-test-remote-development-cbcce41978320fe4'
creates a new workspace and then stops and terminates it
Top 1 slowest examples (650.06 seconds, 100.0% of total time):
Create Remote Development when setup is done in runtime behaves like workspaces actions creates a new workspace and then stops and terminates it
650.06 seconds ./qa/specs/features/shared_examples/create_and_terminate_workspace_shared_examples.rb:5
Finished in 10 minutes 50 seconds (files took 5.68 seconds to load)
1 example, 0 failures
Randomized with seed 2590
Because there's new data-testid
s being introduced here and the test can only be run against Staging, a breakpoint will need to be introduced right after the new workspace listed and to manually add the data-testid
required to locate these elements.
The full list of variable and their values to run this spec can be found in 1password "Run workspaces tests against staging"
bundle exec bin/qa Test::Instance::All https://staging.gitlab.com -- -- qa/specs/features/ee/browser_ui/3_create/remote_development/workspace_actions_spec.rb
Numbered steps to set up and validate the change are strongly suggested.
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.