Discussion: What options do we have for interfacing Release-Tools with our Kubernetes Deployment?
Currently release-tools doesn't have any way to directly interface with a Kubernetes deployment. This is a big shift from where we are now. Utilize this issue to come up with various options that we want to achieve. After we decide on a method for which we can perform this, update epic &683 with our choice and the plan out the issues that get us there.
Items to keep track of when it comes to thinking about potential obsticles
- How do we ensure that we update ONLY the version of GitLab that we want
- How can we ensure that we do not accidentally cause deployments of items that are included from other sources
- How do we prevent dynamic items from being changed in the deployment
- How do we validate that we are not going to stomp on other deployments (we use CI feature
resource_groups
for this today)
Options
Proposal 1
Instead of having release-tools call the k8s-workloads/gitlab-com
repo (via deployer) to roll out auto-deploy changes, we instead modify release-tools to talk to either Kubernetes and/or helm directly (inside all our clusters), and directly call the helm upgrade commands needed to deploy a new version of Gitlab. Currently we leverage calling helm as a whole, substituting only the version of GitLab to be deployed. So this option would allow us to prevent deploying some items which are dynamic in nature or where we shell out for some values that are stored outside of the repository.
Currently if you run the following in an environment
helm -n gitlab upgrade --reuse-values --set global.gitlabVersion=ver_from_auto_deploy gitlab
That will essentially have the desired effect for an auto-deploy. It will change the version of the container image being used for rails services to the version specified, but leave all other values (that come from the configuration state) exactly the same as they are currently.
Proposed workflow:
sequenceDiagram
participant rt as release-tools
participant D as Deployer
participant dt as deploy-tooling
participant gc as gitlab-com
participant k8sa as alpha Kubernetes clusters
participant k8sb as beta Kubernetes clusters
rt->>D: begin Auto-Deploy
D->>dt: start
activate dt
dt-->>dt: Prep
dt-->>dt: Virtual Machines (gitaly/prefect)
dt->>rt: VM deploy complete
deactivate dt
rt->> gc: Confirm no pipelines are running, and lock new pipelines starting
rt->>k8sa: `helm -n gitlab upgrade --reuse-values gitlab`
activate k8sa
k8sa->>rt: helm upgrade complete
deactivate k8sa
rt->>k8sb: `helm -n gitlab upgrade --reuse-values gitlab`
activate k8sb
k8sb->>rt: helm upgrade complete
deactivate k8sb
rt->>gc: Unlock so new pipelines can continue
rt-->>dt: post deploy migrations
dt->>D: done
D->>rt: done Auto-Deploy
Proposal N
Milestones
-
Come up with various options -
DISCUSS -
Choose an option