Enhance e2e test coverage to support Kubernetes executor and Windows node on GKE
Currently, there are no e2e tests for Kubernetes executor on a Windows node in GKE. Manual testing is conducted using the provided README, which outlines the necessary gcloud commands.
Proposal
- Extend the GRIT GKE module to provision a GKE cluster with Windows Nodes
- Add support for the Kubernetes section in the config.toml file used for testing
- Execute a basic "Hello World" test, at least initially
Use cases
Two use cases can be considered:
- Each test is run on a brand new cluster, which can make the job longer due to the need to pull Windows images (Helper and Job image) repeatedly.
- Alternatively, tests can be conducted on a persistent cluster to reduce the time spent pulling images. The cluster can be reprovisioned when a significant change is made, possibly at the GRIT level.
gcloud commands currently used
GKE Cluster Creation
gcloud container clusters create <CLUSTER_NAME> --disk-type=<DISK_TYPE> --disk-size=<DISK_SIZE> --machine-type=<MACHINE_TYPE> --num-nodes=<NUMBER_OF_NODES> --zone=<ZONE>
with:
- CLUSTER_NAME: the cluster name
- DISK_TYPE: the type of the disk -
pd-standard
,pd-ssd
,pd-balanced
, etc ... - DISK_SIZE: the size of the disk
- MACHINE_TYPE: the type of machine. For light used, we recommand using the
g1-small
- NUMBER_OF_NODES: the number of nodes in the cluster default pool (defaults to 3)
- ZONE: the cluster zone
Windows Pool creation
gcloud container node-pools create <POOL_NAME> --disk-type=<DISK_TYPE> --disk-size=<DISK_SIZE> --image-type=WINDOWS_LTSC_CONTAINERD --machine-type=<MACHINE_TYPE> --num-nodes=<NUMBER_OF_NODES> --zone=<ZONE> --cluster=<CLUSTER_NAME>
with:
- POOL_NAME: the name of the windows pool e.g.
windows-pool
- DISK_TYPE: the type of the disk -
pd-standard
,pd-ssd
,pd-balanced
, etc ... - DISK_SIZE: the size of the disk. For windows machine, minimum value allowed is
40GB
- MACHINE_TYPE: the type of machine. For light used, we recommand using the
e2-standard-2
- NUMBER_OF_NODES: the number of nodes in the cluster default pool (defaults to 3). We recommand using
1
- ZONE: the cluster zone
- CLUSTER_NAME: the name of the cluster previously created
Template config.toml
concurrent = 1
check_interval = 1
[session_server]
session_timeout = 1800
[[runners]]
url = "https://gitlab.com/"
executor = "kubernetes"
shell = "pwsh/powershell"
[runners.kubernetes]
image = "xxx"
helper_image = "xxx"
poll_timeout = 3600
[runners.kubernetes.node_selector]
"kubernetes.io/arch" = "amd64"
"kubernetes.io/os" = "windows"
"node.kubernetes.io/windows-build" = "xxx"
[runners.kubernetes.dns_config]
[runners.kubernetes.container_lifecycle]