## Problem to solve As a Platform Engineer / DevOps Engineer, I want to manage my cluster creation process as code, so that everything is under version control. As a CTO, I want to use a tool that supports various providers and is easily extendable with new providers. ## Intended users * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) @nagyv-gitlab run dozens of interviews around Kubernetes and Terraform usage in the past months, and every participant uses Terraform to set up and tear down their clusters, and related cloud project and resources. ## User experience goal - The user should be able to create and own a cluster in a transparent way using GitLab - The user should be able to edit the cluster provided by GitLab ## Proposal ### MVC 1. Provide a project template to create a GKE and AWS cluster as they are provisioned today with GitLab - use the GitLab backend for Terraform 1. The provided template should attach the created cluster to a project given to it in a CI variable - the backend file should be derived from the project we attach the cluster to, this way a single project can be used for several clusters - the current user becomes an owner of the project 1. Create a UI to set up a new project with the selected template, and all the necessary `env` variables needed to run the cluster provisioning step 1. Switch the current UI to the new UI - should we provide an infrastructure project to existing users or not? The resulting Terraform repo should be (over)documented. - It should contain a README, - the CI should describe every step and `env` var, - all Terraform files should describe what a given resource is for and what our initial setup is #### Current UI based cluster actions All of these can stay on the UI and be used as they are today. - set / unset GitLab integration - set / unset GitLab managed cluster - install applications - specify a cluster management project - clear the cache - remove integration - remove integration and resources #### Cluster configuration project path How can the user set the cluster configuration project's "path"? - we provide two inputs: - select from the user's groups - text input to provide the project's name - for project- and group level clusters the select is preselected for the project's group - for project level clusters the text input is prefilled with `<project-name>-cluster` - for group level clusters the text input is prefilled with `<group-name>-cluster` - if the pre-filled project exists, we use a counter `<project-name>-cluster-1` #### Removing the cluster What happens if the user removes the cluster using the "remove integration" button? - we keep the cluster configuration project #### Removing the cluster configuration project What happens if the user removes the cluster configuration project? - we keep the cluster and its imported state #### Making the setup user-friendly Is there a nice way to get Google's OAuth work with such a setup or we'd need the user to create a service account for us to use? Or our script would use OAuth to create a service account, and attach that to the new project? #### Edge cases What happens if the user has no Runner attached or run out of Runner minutes? - On the "form' page, we should check if runners are available for the user, and put a huge red notification out if they are not. The notification should state that clusters are created via the runner, they have no runner available, they can still proceed, and have all their code ready, but the cluster won't be created, and its pipeline should be started manually, once runners will be available. What happens if the user removes the Terraform state file, then re-runs the Terraform project? - We would create a new cluster, and attach it to GitLab beside the other cluster. Can a single Terraform project be used to create multiple clusters and attach them to different resources, levels? - If we want to support this use-case, then we should provide a user to choose from existing repos when creates a new cluster - we might support this use-case without a UI. As the details where to import the cluster come from `env` vars, the user can set up a job that uses different variables. ### Later steps - provide a UI to configure the cluster at creation as described in https://gitlab.com/gitlab-org/gitlab/-/issues/29862#note_215822555 ## Further details @bdowney has great repos for EKS, GKE, AKS under https://gitlab.com/gitops-demo/infra The created Terraform project might be configure using input variables. It would be great to iterate towards a "Terraform input UI" where users can run any Terraform module with one-off parameters provided through a UI. Basically, the UI would set `TF_ENV_` variables, and the core feature is to auto-generate the UI based on code inspection. ## Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)?--> ## Documentation The resulting Terraform repo should be (over)documented. - It should contain a README, - the CI should describe every step and `env` var, - all Terraform files should describe what a given resource is for and what our initial setup is <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change * Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements * If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html --> ## Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> ## What does success look like, and how can we measure that? We increase the number of clusters registered with GitLab. ## What is the type of buyer? <!-- What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/ In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#four-tiers --> ## Is this a cross-stage feature? <!-- Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features --> ## Links / references