Enable interactive web terminal for GitLab.com

Problem to solve

We have support for docker executor for web terminals and we should enable it on GitLab.com since it's a core feature.

Further details

This will add the ability to use interactive web terminals on .com, giving many users the ability to take advantage of this feature. We have to be careful before enabling this feature, however, since it can have an impact on production performance and security characteristics. This issue represents the due diligence from an engineering side to do this investigation and, once complete (assuming no hurdles), enable the feature on .com.

Currently, shared runners are using the docker executor, which is partially supported, there are talks to use kubernetes since we have better support for it. From a security perspective both. There are also plans to migrate to kubernets for our runner managers but that will not affect the proposal below or vice versa.

Resoruces

Proposal

This issue picks up after the ~backstage work (steps 1-5) in https://gitlab.com/gitlab-org/gitlab-ce/issues/52611 has been completed. Please see that issue for those steps.

6. Start using private-runners-manager-5.gitlab.com for CI jobs

Given all previous steps have been successful especially step 4 & 5 we can start using the runner manager for running some tests. We can do this in two ways:

  1. Set specific jobs with the correct tags (interactive-web-terminal, gitlab-org) inside of the .gitlab-ci.yml file
  2. Configure the group runner to pick up untagged jobs

It might also be ideal to do both of them, reason being the runner manager will pick up more jobs and still have predictability on which jobs we can find the interactive web terminal for debugging a running job

Requirements/Stakeholders:

  • SRE/~Verify GitLab.com admin to configure runner to pick up untagged jobs
  • ~Verify to add the correct tags to .gitlab-ci.yml if we want specific jobs to run them

7. Roll out other runner managers

Rolling this out to the other runners, following the order below. All of the steps require us to update the respective role inside of https://ops.gitlab.net/gitlab-cookbooks/chef-repo and a period of 3-5 days each to monitor the situation.

  1. Update all the private runner manages by updating the gitlab-runner-prm
  2. Update gitlab shared runner managers by updating gitlab-runner-gsrm
  3. Update the final shared runner manages by updating gitlab-runner-srm

Requirements/Stakeholders:

  • ~Verify To add the necessary configuration

8. Remove private-runners-manager-5.gitlab.com

private-runners-manager-5.gitlab.com served its purpose and can be removed since there is no benefit of having another manager.

Requirements/Stakeholders:

  • ~Verify to update chef configurtion and remove the box.

What does success look like, and how can we measure that?

  • Users can use the web terminal on their jobs when they are GitLab.com user and for the Web IDE.
  • Easy to maintain for SREs.
  • Doesn't affect any uptime for CI.
  • Performance of each runner manager is not effected by a large amount.
Edited by Jason Yavorsky