Support end-to-end QA test in CI for Workspaces

Description

This is a followup to an issue created earlier for supporting a minimal e2e test.The scope of the issue created earlier was to have a working e2e spec that can be run locally but not on CI or a long-lived sandbox environment.

The scope of this ticket is to investigate / implement changes to support an e2e spec in CI/sandbox environment

This effort is granularized into the following

1. Install ingress-controller in the k8s cluster
2. Agent setup in the k8s cluster
3. Installing Workspaces proxy in the k8s cluster
4. Change the current e2e spec to do all the pre-requisties
5. When the workspace is created clicking on the workspace link should load the VSCode WebIDE & Terminal is workable

added Category:Remote Development devopscreate featureaddition groupeditor [DEPRECATED] sectiondev spike typefeature workflowin dev + 1 deleted label

assigned to @hkhanna2

added to epic &9190

cc @oregand

Hi @nprabakaran,

My understanding is that issue will be handed over to you soon. Just to highlight, you may also want to take a look at the work done in an earlier issue to support a local e2e spec (it's disabled in CI however).

Do let me know if I can fill in any context / background info that you may find useful.

cc @oregand

@nprabakaran

Please ping me if anything is needed here!

Sure, thank you @hkhanna2 and @oregand . Will get back if there are any questions!

changed milestone to %16.0

removed 1 deleted label

added Deliverable label

assigned to @nprabakaran

unassigned @hkhanna2

This is definitely not blocking us for the merge to master but is a good to have.

@oregand - Should we move this to Remote Development • Workspaces Beta (&10122 - closed) or keep it in Viable itself?

Let's keep it Viable, I want us to keep the initial scope for Remote Development • Workspaces Beta (&10122 - closed) as minimal as possible 👍

mentioned in issue #389342 (closed)

mentioned in issue gitlab-org/quality/triage-reports#12004 (closed)

Moving this to workflowrefinement and removing Deliverable , until we have a more specific plan on what we need to do to get this running on CI.

added workflowrefinement label and removed Deliverable workflowin dev labels

mentioned in issue #407154 (closed)

added quad-planningcomplete-action label

added groupide label and removed groupeditor [DEPRECATED] label

changed epic to &10387 (closed)

mentioned in merge request !119736 (merged)

changed epic to &10599

changed title from Support end-to-end QA test in CI of remote dev architecture spike to Support end-to-end QA test in CI for Workspaces

changed milestone to %16.1

marked this issue as related to #408389 (closed)

changed the description

mentioned in issue #396854

These are the existing e2e specs with agent config:

1. https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/specs/features/ee/api/7_configure/kubernetes/kubernetes_agent_spec.rb
2. https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/specs/features/browser_ui/7_configure/auto_devops/create_project_with_auto_devops_spec.rb

Also the other spec which involves Oauth application:

1. https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/specs/features/browser_ui/1_manage/login/login_via_oidc_with_gitlab_as_idp_spec.rb

added rd-workflowunprioritized label

set weight to 5

marked this issue as related to #414915 (closed)

added workflowin dev label and removed workflowrefinement label

changed milestone to %16.2

added missed:16.1 label

marked this issue as related to #418988 (closed)

mentioned in merge request !123454 (merged)

changed milestone to %16.3

added missed:16.2 label

marked this issue as related to #420316 (closed)

marked this issue as related to #420405

@hkhanna2 this issue had a specific milestone but did not have a iteration, so the milestone has been set back to %Next 1-3 releases.

changed milestone to %Next 1-3 releases

@nprabakaran

Can you refine this issue using the Remote Development issue template.

• https://about.gitlab.com/handbook/engineering/development/dev/create/ide/#2-pre-iteration-planning-meeting

@oregand I remember we previously had a discussion (I think in a retro?) about whether or not QA issues should be part of our standard remote development workflow or not.

I seem to remember that the conclusion was that they should not, because the QA team members are not fully dedicated to the Remote Development team, and therefore their work should not be included and calculated as part of our team's velocity.

We can look for the discussion, but I think that was the conclusion.

However, it should be noted that the issues that @shekharpatnaik owns ARE managed as part of our workflow, even though he is not 100% dedicated to our team. And also @ealcantara splits his effort with Category:Web IDE.

So, from that perspective, our velocity is already inherently inaccurate due to them not being 100% dedicated on the team (although we can evolve our process in the future to account for this).

But I also believe they are in a different situation because their assigned features/issues are directly related to the product, require awareness and discussion of the full team during IPMs, and may be blockers to other issues. Thus they warrant being handled as part of our standard remote development workflow.

In any case, we need to decide how to handle QA issues, especially in light of our new automations.

---

PROPOSAL 1:

1. Any issues that QA owns and is working on should have the QA label assigned, in addition to the Category:Remote Development label.
2. All of the remote development workflow bot automation rules should be updated to exclude any issues which have the QA label. I.e., it will not touch them or make any automated updates or comments related to milestones, iterations, labels, etc.

The benefits of this proposal is that we are not imposing our workflow on the QA team members, i.e. as in this case by asking them to submit an issue refinement, and manage iteration/milestone/etc.

One slight downside of this approach is that these QA issues will then not show up on our iteration planning board, because they are not in any ~rd-workflow::* label.

---

PROPOSAL 2:

1. Alternately, we could require that the QA issues be included and managed according to our remote development team workflow.

The downside of this is as noted above, it places requirements on QA team members to follow our process, which may not provide value for them in their workflow.

---

What are your thoughts on this, @nprabakaran ?

And WDYT @oregand and @ericschurter ?

Personally, my vote is for PROPOSAL 1, to make life easier for the QA team members, and the slight downside is not a concern.

When we decide, I can update the process in the handbook to reflect it.

cc @vtak @hkhanna2 @a_akgun

One other benefit of PROPOSAL 1 is that we can use this same process precedent (including automation rules) for other types of issues that we don't necessarily want to see on our iteration planning board, such as the security issues that may linger open for many weeks.

We might even want to introduce a new generic ~rd-workflow::ignore state to be used for this purpose, which would allow us to write the automation rules in a generic way. And we could also even show this column on the iteration board if we wanted to have visibility of them there, but outside the standard process flow.

@nprabakaran

Are you happy with PROPOSAL 1? i.e. we add the QA label to any issues you are working on and we can exclude them from the Category:Remote Development automation? I agree with @cwoolley-gitlab this is the best path forward.

@nprabakaran @cwoolley-gitlab

We might even want to introduce a new generic ~rd-workflow::ignore state to be used for this purpose, which would allow us to write the automation rules in a generic way. And we could also even show this column on the iteration board if we wanted to have visibility of them there, but outside the standard process flow.

This actually seems like the best choice, lets's just mark these as ~rd-workflow::ignore and our automation can ignore those items.

But I also believe they are in a different situation because their assigned features/issues are directly related to the product, require awareness and discussion of the full team during IPMs, and may be blockers to other issues.

This is a great point @cwoolley-gitlab and that's the tricky distinction.

I also support a ~rd-workflow::ignore label that we use for automation, as it resolves most of my other concerns about the meta issues that get assigned to me as well.

@cwoolley-gitlab @oregand This PROPOSAL 1 seems great where we can include QA labels and also add the ~rd-workflow::ignore so that it can be on the board if needed but does not affect the iteration.

Thank you Chad for this proposal which would really make it easier for the QA team members to exclude from the refinement process

The MR for this change - !123454 (merged)

mentioned in merge request gitlab-org/quality/triage-ops!2350 (merged)

This issue can be closed once this MR to add variables is completed - https://gitlab.com/gitlab-org/quality/engineering-productivity-infrastructure/-/merge_requests/436

added rd-workflowignore label and removed rd-workflowunprioritized label

mentioned in test case #396854

closed